Method and apparatus for encoding video by compensating for pixel value according to pixel groups, and method and apparatus for decoding video by the same

ABSTRACT

Provided are a method and apparatus of encoding a video by compensating for a pixel value and a method and apparatus of decoding a video by compensating for a pixel value. The method of encoding the video includes: encoding image data; decoding the encoded image data and generating a restored image by performing loop filtering on the decoded image data; determining a compensation value corresponding to errors between a predetermined group restored pixels in the restored image and corresponding original pixels, and a pixel group including a restored pixel to be compensated for by using the compensation value; and encoding the compensation value and transmitting the encoded compensation value and a bitstream of the encoded image data.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application claims priority from Korean Patent Application No.10-2010-0031143, filed on Apr. 5, 2010 in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein in itsentirety by reference.

BACKGROUND

1. Field

Apparatuses and methods consistent with exemplary embodiments relate toencoding and decoding a video.

2. Description of the Related Art

As hardware for reproducing and storing high resolution or high qualityvideo content is being developed and supplied, a need for a video codecfor effectively encoding or decoding the high resolution or high qualityvideo content is increasing. In a related art video codec, a video isencoded according to a limited encoding method based on a macroblockhaving a predetermined size.

The quality of an image may be distorted due to encoding and decoding ofa video, and a post-processing module may be added to a decoder or arestored image so as to improve the quality of the restored image.

SUMMARY

One or more exemplary embodiments provide a method and apparatus forencoding video, and a method and apparatus for decoding video, forcompensating for a pixel value of a predetermined pixel group.

According to an aspect of an exemplary embodiment, there is provided amethod of encoding a video for compensating for a pixel value, themethod including: encoding image data; decoding the encoded image dataand generating a restored image by performing loop filtering on thedecoded image data; determining a compensation value corresponding toerrors between a predetermined group of restored pixels in the restoredimage and corresponding original pixels, and a pixel group including arestored pixel to be compensated for by using the compensation value;and encoding the compensation value and transmitting the encodedcompensation value and a bitstream of the encoded image data.

The determining the compensation value and the pixel group may include:determining an extreme value level indicating a degree of closeness to amaximum value or a minimum value according to restored pixels bycomparing pixel values of neighboring restored pixels in the restoredimage; and determining the pixel group comprising the restored pixel tobe compensated for among the neighboring restored pixels based on thedetermined extreme value level according to restored pixels.

The determining the pixel group based on the extreme value level mayinclude classifying the neighboring restored pixels into pixel groupsincluding restored pixels having a same extreme value level based on thedetermined extreme value level according to restored pixels, anddetermining a pixel group of at least one extreme value level as thepixel group including the restored pixel to be compensated for, and thedetermining the compensation value and the pixel group may furtherinclude determining the compensation value for the determined pixelgroup of the at least one extreme value level.

The determining the compensation value and the pixel group may include:classifying restored pixels of the restored image into pixel groupsincluding restored pixels in a same band, according to bands, obtainedby dividing a total section of pixel values; and determining thecompensation value according to the pixel groups according to bands.

The classifying the restored pixels according to bands may includeclassifying the restored pixels into the pixel groups according tobands, based on bands obtained by dividing the total section of thepixel values into a number of 2 to the power of a positive integer.

An index of the number of 2 to the power of the positive integer may bedetermined based on a number of most significant bits in a bit depth ofthe restored pixels.

The total section of the pixel values may be within a range of anextended bit depth.

The determining the compensation value and the pixel group may include:classifying restored pixels of the restored image into pixel groupsincluding restored pixels disposed on a same line, according to lines;and determining the compensation value according to the pixel groupsaccording to lines.

The classifying the restored pixels into the pixel groups according tolines may include detecting restored pixels forming lines in at leastone of a horizontal direction, a vertical direction, a diagonaldirection, a curved direction and a boundary direction of apredetermined object, from among the restored pixels of the restoredimage.

The determining the compensation value and the pixel group may includedetermining the compensation value by using an average value of errorsbetween restored pixels of the pixel group and corresponding originalpixels.

The determining the compensation value and the restored pixel mayinclude determining the compensation value for all restored pixels to becompensated for, or individually determining the compensation valueaccording to predetermined groups of the restored pixels to becompensated for.

The generating the restored image may be performed by using adaptiveloop filtering using a plurality of continuous one-dimensional filters.

The determining the compensation value and the pixel group may includedetermining the compensation value and the restored pixel to becompensated for according to at least one data unit from among an imagesequence, a slice, a frame, and a coding unit of the input video.

The transmitting the bitstream may include inserting and transmittingthe encoded compensation value into a slice header.

The encoding the input image sequence may include: dividing a pictureinto a maximum coding unit; performing encoding on at least one deepercoding unit according to depth, according to regions split byhierarchically dividing the maximum coding unit as the depth deepens todetermine an encoding mode of a coding unit of a coded depth, theencoding mode including information of at least one coded depth thatgenerates a least encoding error; and outputting the encoded image dataaccording to the determined coded depth and encoding mode.

According to an aspect of another exemplary embodiment, there isprovided a method of decoding a video for compensating for a pixelvalue, the method including: extracting encoded image data and acompensation value from a bitstream by parsing the bitstream of anencoded image; decoding the extracted image data and generating arestored image by performing loop filtering on the decoded image data;determining a pixel group including a restored pixel to be compensatedfor from among restored pixels of the restored image, by using theextracted compensation value; and compensating for an error between therestored pixel of the determined pixel group and a correspondingoriginal pixel by using the extracted compensation value.

The determining the pixel group may include: determining an extremevalue level, which indicates a degree of closeness to a maximum value ora minimum value according to the restored pixels by comparing pixelvalues of neighboring restored pixels in the restored image; anddetermining the pixel group comprising the restored pixel to becompensated for among the neighboring restored pixels based on thedetermined extreme value level.

The determining the extreme value level may include classifying theneighboring restored pixels into pixel groups including restored pixelshaving a same extreme value level based on the determined extreme valuelevel, and determining a pixel group of at least one extreme value levelas the pixel group including the restored pixel to be compensated for.

The determining the pixel group may include classifying the restoredpixels of the restored image into pixel groups according to bands.

The compensating for the error may include compensating for errors ofpixel values of the restored pixels of the pixel groups according toextreme value levels by using the compensation values according toextreme value levels for compensating for the pixel values of the pixelgroups according to the extreme value levels.

The determining the pixel group may include classifying the restoredpixels of the restored image into pixel groups including restored pixelsdisposed on a same line, according to lines, and determining a pixelgroup, from among the pixel groups according to lines, as the pixelgroup comprising the restored pixel to be compensated for.

The compensating for the error may include compensating for pixel valuesof the restored pixels in the pixel groups according to lines, by usingcompensation values of the pixel groups according to lines forcompensating for the pixel values of the pixel groups according tolines.

The classifying the restored pixels into the pixel groups according tolines may include detecting restored pixels forming lines in at leastone of a horizontal direction, a vertical direction, a diagonaldirection, a curved direction and a boundary direction of apredetermined object, from among the restored pixels of the restoredimage.

The compensation value may be determined by using an average value oferrors between the restored pixels of the pixel group and correspondingoriginal pixels during encoding of the encoded image data.

The compensating for the error may include compensating for all restoredpixels to be compensated for by using the compensation value.

The compensating for the error may include compensating for the pixelvalues of the restored pixels by using compensation values individuallydetermined according to predetermined groups of the restored pixels tobe compensated for.

The generating the restored image may be performed by using adaptiveloop filtering using a plurality of continuous one-dimensional filters.

The encoded image data may be encoded by dividing a picture into amaximum coding unit, and performing encoding on at least one deepercoding unit according to depth, according to regions split byhierarchically dividing the maximum coding unit as the depth deepens todetermine information of an encoding mode of a coding unit of a codeddepth, which includes information about at least one coded depth thatgenerates a least encoding error, from the bitstream, and the generatingthe restored image may include decoding the image data encoded based onthe coded depth and the encoding mode, based on the information aboutthe encoding mode, and performing the loop filtering.

According to an aspect of another exemplary embodiment, there isprovided an apparatus for encoding a video for compensating for a pixelvalue, the apparatus including: an encoder which encodes image data; arestored image generator which decodes the encoded image data andgenerates a restored image by performing loop filtering on the decodedimage data; a compensation value and pixel group determiner whichdetermines a compensation value corresponding to errors between apredetermined group of restored pixels and corresponding originalpixels, and a pixel group including a restored pixel to be compensatedfor by using the compensation value; and a transmitter which encodes thecompensation value and transmits the encoded compensation value and abitstream of the encoded image data.

According to an aspect of another exemplary embodiment, there isprovided an apparatus for decoding a video for compensating for a pixelvalue, the apparatus including: an extractor which extracts encodedimage data and a compensation value from a bitstream by parsing thebitstream of an encoded image; a restored image generator which decodesthe extracted image data and generates a restored image by performingloop filtering on the decoded image data; a pixel group determiner whichdetermines a pixel group including a restored pixel to be compensatedfor from among restored pixels of the restored image, by using theextracted compensation value; and a restored pixel compensator whichcompensates for an error between the restored pixel of the determinedpixel group and a corresponding original pixel by using the extractedcompensation value.

According to an aspect of another exemplary embodiment, there isprovided a computer readable recording medium having recorded thereon aprogram for executing any one of the above-described methods.

BRIEF DESCRIPTION-OF THE DRAWINGS

The above and/or other aspects will become more apparent by describingin detail exemplary embodiments with reference to the attached drawingsin which:

FIG. 1 is a block diagram of an apparatus for encoding a video forcompensating for a pixel value, according to an exemplary embodiment;

FIG. 2 is a block diagram of an apparatus for decoding a video forcompensating for a pixel value, according to an exemplary embodiment;

FIG. 3 illustrates neighboring restored pixels to be compared to arestored pixel so as to determine an extreme value level of the restoredpixel, according to an exemplary embodiment;

FIG. 4 is a flowchart for describing adaptive loop filtering accordingto an exemplary embodiment;

FIG. 5 is a flowchart for describing adaptive loop filtering accordingto another exemplary embodiment;

FIG. 6 is a flowchart illustrating a method of encoding a video forcompensating for a pixel value, according to an exemplary embodiment;

FIG. 7 is a flowchart illustrating a method of decoding a video bycompensating for a pixel value, according to an exemplary embodiment;

FIG. 8 is a block diagram of an apparatus for encoding a video bycompensating for a pixel value after performing loop filtering based oncoding units having a tree structure, according to an exemplaryembodiment;

FIG. 9 is a block diagram of an apparatus for decoding a video bycompensating for a pixel value after performing loop filtering based oncoding units having a tree structure, according to an exemplaryembodiment;

FIG. 10 is a diagram for describing a concept of coding units accordingto an exemplary embodiment;

FIG. 11 is a block diagram of an image encoder based on coding unitsaccording to an exemplary embodiment;

FIG. 12 is a block diagram of an image decoder based on coding unitsaccording to an exemplary embodiment;

FIG. 13 is a diagram illustrating deeper coding units according todepths, and partitions, according to an exemplary embodiment;

FIG. 14 is a diagram for describing a relationship between a coding unitand transformation units, according to an exemplary embodiment;

FIG. 15 is a diagram for describing encoding information of coding unitscorresponding to a coded depth, according to an exemplary embodiment;

FIG. 16 is a diagram of deeper coding units according to depths,according to an exemplary embodiment;

FIGS. 17 through 19 are diagrams for describing a relationship betweencoding units, prediction units, and transformation units, according toan exemplary embodiment;

FIG. 20 is a diagram for describing a relationship between a codingunit, a prediction unit or a partition, and a transformation unit,according to encoding mode information of Table 2;

FIG. 21 is a flowchart illustrating a method of encoding a video bycompensating for a pixel value after performing loop filtering based oncoding units having a tree structure, according to an exemplaryembodiment; and

FIG. 22 is a flowchart illustrating a method of decoding a video bycompensating for a pixel value after performing loop filtering based oncoding units having a tree structure, according to an exemplaryembodiment.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, exemplary embodiments will be described more fully withreference to the accompanying drawings. Expressions such as “at leastone of,” when preceding a list of elements, modify the entire list ofelements and do not modify the individual elements of the list.

A method and apparatus for encoding a video, and a method and apparatusfor decoding a video, by compensating for errors of pixel values of apredetermined pixel group, according to exemplary embodiments will nowbe described with reference to FIGS. 1 through 22. In detail, encodingand decoding a video by compensating for a pixel value after performingloop filtering, according to exemplary embodiments will be describedwith reference to FIGS. 1 through 7, and encoding and decoding a videofor compensating for a pixel value after performing loop filtering basedon coding units having a tree structure, according to exemplaryembodiments will be described with reference to FIGS. 8 through 22.

Hereinafter, encoding and decoding of a video by compensating a pixelvalue after performing loop filtering according to exemplary embodimentswill be described with reference to FIGS. 1 through 7.

FIG. 1 is a block diagram of a video encoding apparatus 10 forcompensating for a pixel value, according to an exemplary embodiment.

The video encoding apparatus includes an encoder 12, a restored imagegenerator 14, a compensation value and pixel group determiner 16, and atransmitter 18. The operations of the encoder 12, the restored imagegenerator 14, and the compensation value and pixel group determiner 16of the video encoding apparatus 10 may be organically controlled by avideo encoding processor, a central processor, a graphic processor, orthe like.

The encoder 12 encodes an image in a picture unit from among a sequenceof input images. The encoder may generate encoded image data byperforming motion estimation, inter prediction, intra prediction,transformation, and quantization on an input image.

The encoder 12 may use any video encoding method, such as MPEG 1, 2, 4,and H.26x. For example, the encoder 12 may use a method of videoencoding based on coding units having a tree structure, according to anexemplary embodiment, which will be described later with reference toFIGS. 8 through 22.

The restored image generator 14 may receive the image data encoded bythe encoder 12, and generate a restored image by decoding the encodedimage data and performing loop filtering on the decoded image data.

The restored image generator 14 may generate the decoded image data byperforming inverse quantization, inverse transformation, interprediction, motion compensation, and intra prediction on the encodedimage data.

The decoding performed on the encoded image data by the restored imagegenerator 14 may be performed as inverse processes of a video encodingmethod performed by the encoder 12. For example, the video encodingapparatus 10 when the encoder 12 and the restored image generator 14performs the method of encoding a video, according to an exemplaryembodiment, will be described later with reference to FIGS. 8 through22.

The restored image generator 14 may perform in-loop filtering on thedecoded image data. The in-loop filtering may selectively includedeblocking filtering and adaptive loop filtering. The adaptive loopfiltering may be performed by using a plurality of continuousone-dimensional filters. The adaptive loop filtering according toexemplary embodiments will be described in detail later with referenceto FIGS. 4 and 5.

The compensation value and pixel group determiner 16 receives the inputimage and the restored image output by the restored image generator 14,determines a compensation value about an error between each restoredpixel of a predetermined group in the restored image, and acorresponding original pixel in the input image, and determines a pixelgroup including restored pixels to be compensated for by using thecompensation value.

The compensation value and pixel group determiner 16 compares pixelvalues of neighboring restored pixels from among the restored pixels inthe restored image, and determines an extreme and/or edge value levelindicating a closeness to a maximum value and a minimum value.Hereinafter, for convenience of explanation, an ‘extreme and/or edgevalue level’ may represent at least one of an edge value level and anedge value level. The compensation value and pixel group determiner 16may determine the neighboring restored pixels into pixel groupsincluding restored pixels having the same extreme and/or edge valuelevel, based on each extreme and/or edge value level of the neighboringrestored pixels.

The compensation value and pixel group determiner 16 may determine atleast one pixel group of extreme and/or edge value level from among theclassified pixel groups as a pixel group having pixel values to becompensated for. The compensation value and pixel group 16 may determineto compensate for pixel values of pixel groups of minimum and maximumextreme and/or edge value levels, or pixel values of pixel groups ofextreme and/or edge value levels in a predetermined range. A method ofdetermining a target to be compensated for based on extreme and/or edgevalue levels of neighboring restored pixels will be described later withreference to FIG. 3.

The compensation value and pixel group determiner 16 may determine tocompensate for pixel values of pixel groups according to bands. Thecompensation value and pixel group determiner 16 may split a total rangeof pixel values of the restored pixels into a plurality of split bandsso as to assign pixel groups of the restored pixels. The compensationvalue and pixel group determiner 16 may classify restored pixels in thesame band into pixel groups according to bands, based on the pixelvalues of the restored pixels. Here, all pixel values of the restoredpixels in the pixel group according to bands may be determined to becompensated for, and the compensation value and pixel group determiner16 may separately determine a compensation value for each pixel groupaccording to bands.

For high-speed processing, the total range of the pixel values may besplit into a number of bands identical to a number of 2 to the power ofa positive integer. For high-speed processing, when a number of mostsignificant bits in a bit depth of a bit string of the restored pixelsis p, the total range of the pixel values may be split into a number ofbands identical to a number of 2^(p). Alternatively, the total range ofthe pixel values may be identical to a range of extended bit depth ofthe restored pixel.

The compensation value and pixel group determiner 16 may analyze therestored image, detect lines in a predetermined direction, and classifythe restored pixels into pixel groups according to lines includingrestored pixels on the same line. When lines in various directions, suchas a horizontal direction, a vertical direction, a diagonal direction, acurved direction and a boundary direction of a predetermined object, aredetected, pixels forming each line may be classified into one pixelgroup. The compensation value and pixel group determiner 16 mayindividually determine a compensation value for each pixel groupaccording to lines.

The compensation value and pixel group determiner 16 may determine anaverage value of errors between the restored pixels to be compensatedfor and corresponding original pixels as the compensation value. Theerror between the restored pixel and the original pixel may include adifference between the restored pixel and the original pixel, anabsolute value of the difference, or a square of the difference. Thecompensation value and pixel group determiner 16 may determine onecompensation value to be equally applied to the entire restored pixelsto be compensated for, or individually determine a compensation valueaccording to pixel groups classified according to characteristics.

The compensation value and pixel group determiner 16 may determine therestored pixel to be compensated for and determine a correspondingcompensation value, according to at least one data unit from among animage sequence, a slice, a frame, and a coding unit of the input video.

The transmitter 18 receives and encodes the compensation valuedetermined by the compensation value and pixel group determiner 16. Thetransmitter 18 receives the image data encoded by the encoder 12, andgenerates and outputs a bitstream including the encoded compensationvalue and the encoded image data. The encoded image data may beconverted into a bitstream format via entropy encoding, and insertedinto a bitstream for transmission.

The transmitter 18 may receive additional information about a method ofdetermining a pixel group from the compensation value and pixel groupdeterminer, and encode and insert the additional information into thebitstream. Since the method may be based on the extreme and/or edgevalue levels, bands, or lines as described above, information indicatinghow the compensation value is employed and a pixel group using thecompensation value may be transmitted.

When the restored image generator 14 performs the adaptive loopfiltering, the transmitter 18 may receive information about a loopfilter coefficient for the adaptive loop filtering, and encode andinsert the information into the bitstream. The video encoding apparatus10 may split the image to parts of a square shape, a rectangular shapeor even irregular shape and perform selective correction for onlyspecific groups of pixels in concrete region. Based on the split partsof the image, pixel values may be compensated adaptively to the contentsof the image. Also, the video encoding apparatus 10 may transmitinformation regarding groups of pixels to be corrected by explicitsignaling and implicit signaling.

The video encoding apparatus 10 may provide the information about thecompensation value obtained during encoding to a decoder so that thedecoder is able to support a post-process that may be performed toreduce an error between the restored image and the original image. Also,since the compensation value is determined according to the pixelgroups, an amount of transmission bits may be reduced by only encodingand transmitting the information about the compensation value, withouthaving to encode and transmit information about a location of individualpixels.

FIG. 2 is a block diagram of a video decoding apparatus 20 forcompensating for a pixel value, according to an exemplary embodiment.

The video decoding apparatus 20 includes an extractor 22, a restoredimage generator 24, a pixel group determiner 26, and a restored pixelcompensator 28. The operations of the extractor 22, the restored imagegenerator 24, the pixel group determiner 26 and the restored pixelcompensator 28 of the video decoding apparatus 20 may be organicallycontrolled by a video decoding processor, a central processor, a graphicprocessor, or the like.

The extractor 22 receives and parses a bitstream about an encoded image,and extracts encoded image data and information related to acompensation value from the bitstream. The information related to thecompensation value may include information about the compensation value.When the information related to the compensation value further includesinformation about a method of determining a pixel group to becompensated for by using the compensation value, the extractor 22 mayextract the compensation value and the information about the method fromthe bitstream. The extractor 22 may extract at least one of thecompensation value and the information related to the compensation valueaccording to at least one data unit from among an image sequence, aslice, a frame, and a coding unit of an input video.

The extractor 22 may extract encoding information, such as an encodingmethod and an encoding mode, which is used to decode the encoded imagedata. When information about a loop filter coefficient for adaptive loopfiltering is inserted into the bitstream, the extractor 22 may extractthe information about the loop filter coefficient from the bitstream.

The restored image generator 24 receives the encoded image data, theencoding information, and the information about the loop filtercoefficient, which are extracted by the extractor 22, and generates arestored image by decoding the encoded image data and performing loopfiltering on the decoded image data.

The decoding of the encoded image data may be performed as inverseprocesses of a method of video encoding performed on the encoded imagedata. For example, when the encoded image data is encoded andtransmitted according to a method of video encoding based on codingunits having a tree structure, according to an exemplary embodiment, therestored image generator 24 may decode the encoded image data accordingto a method of video decoding based on coding units having a treestructure.

The restored image generator 24 may selectively perform in-loopfiltering, such as deblocking filtering and adaptive loop filtering, onthe decoded image data. The adaptive loop filtering may be performed byusing a plurality of continuous one-dimensional filters.

The pixel group determiner 26 may receive the restored image generatedby the restored image generator 24, and the information related to thecompensation value extracted by the extractor 22, and determine a pixelgroup including restored pixels to be compensated for by using thecompensation value, from among restored pixels of a predetermined groupin the restored image. The restored pixel compensator 28 receives thecompensation value extracted by the extractor 22, and the informationabout the pixel group determined by the pixel group determiner 26, andcompensates for the pixel values of the restored pixels by using thecompensation value and outputs the restored image having the restoredpixel values.

When the information about the method of determining the pixel group isextracted by the extractor 22, the pixel group determiner 26 maydetermine the pixel group having the pixel values to be compensated forby using the method. For example, the pixel group determiner 26 maydetermine whether to classify the restored pixels according to extremeand/or edge value levels, bands of pixel values, or lines, and determinethe pixel group based on the method.

The pixel group determiner 26 may determine an extreme and/or edge valuelevel according to restored pixels by comparing pixel values ofneighboring restored pixels in the restored image. The pixel groupdeterminer 26 may classify the neighboring restored pixels based on theextreme and/or edge value levels, and determine a pixel group includingthe restored pixels of at least one predetermined extreme and/or edgevalue level as a pixel group including restored pixels having pixelvalues to be compensated for by using the compensation value. Therestored pixel compensator 28 may compensate the pixel values of therestored pixels in the determined pixel group, by using the compensationvalue.

Alternatively, the pixel group determiner 26 may classify the restoredpixels in the restored image into pixel groups according to bands, basedon bands obtained by splitting a total section of the pixel values. Therestored pixel compensator 28 may compensate for the pixel values of therestored pixels in the pixel group according to bands, by using acompensation value according to bands for the restored pixels in thepixel group according to bands.

The total section of the pixel values may be split into a number ofbands equal to a number of 2 to the power of a positive integer. Here,an index of the number of 2 to the power of a positive integer may bedetermined based on a number of most significant bits in a bit depth ofthe restored pixels. Also, the total section of the pixel values may bea range of extended bit depth of the restored pixels.

The pixel group determiner 26 may classify the restored pixels of therestored image into pixel groups according to lines. The restored pixelcompensator 28 may compensate for pixel values of the restored pixels inthe pixel group according to lines, by using a compensation value forthe pixel group according to lines. The pixel group determiner 26 maydetect restored pixels forming lines in at least one horizontaldirection, a vertical direction, a diagonal direction, a curveddirection or a boundary direction of a predetermined object, from amongthe restored pixels of the restored image.

The compensation value may be determined and transmitted by using anaverage value of errors between the restored pixels and correspondingoriginal pixels during encoding. The restored pixel compensator 28 maycompensate for all pixel values of the restored pixels to be compensatedfor by using one compensation value. Alternatively, when thecompensation value extracted by the extractor 22 is set according topixel groups, the restored pixel compensator 28 may compensate for thepixel values by using the compensation value individually determinedaccording to the pixel groups.

The video encoding apparatus 10 and the video decoding apparatus 20 maycompensate for a systematic error generated between a restored image andan original image when an encoded image is decoded and restored. Thevideo encoding apparatus 10 may transmit information regarding groups ofpixels to be corrected by explicit signaling and implicit signaling. Thevideo encoding apparatus 10 and the video decoding apparatus 20 maysplit the image to parts of a square shape, a rectangular shape or evenirregular shape and perform selective correction for only specificgroups of pixels in concrete region. Based on the split parts of theimage, pixel values may be compensated adaptively to the contents of theimage.

As an example of the systematic error between the restored image and theoriginal image, the average value of the errors of pixel values betweenrestored pixels in a predetermined group and corresponding originalpixels may not be 0. Accordingly, the video encoding apparatus 10 andthe video decoding apparatus 20 compensates for the error between therestored pixel and the original pixel.

The compensation value and pixel group determiner 16 may determine acompensation value according to Equation 1 below.

$\begin{matrix}{{corr} = \frac{\sum\limits_{m = 1}^{M}\left( {{{Org}\left( {x_{m},y_{m}} \right)} - {{Rec}\left( {x_{m},y_{m}} \right)}} \right)}{M}} & {{Equation}\mspace{14mu} 1}\end{matrix}$

Here, m denotes an integer from 1 to M, and an average value corr oferrors between pixel values Org(x_(m), y_(m)) of an original pixel andpixel values Rec(x_(m), y_(m)) of a restored pixel may be used as acompensation value of pixel groups {(x_(m), y_(m))} including pixels aslocations (x_(m), y_(m)).

The restored pixel compensator 28 may compensate for pixel values ofrestored pixels in a pixel group according to Equation 2 below.

Rec _(corrected)(x _(m) ,y _(m))=Rec(x _(m) ,y _(m))+corr  Equation 2

The restored pixel compensator 28 may compensate for the pixel valuesRec(x_(m), y_(m)) of the restored pixel by using the compensation valuecorr, and output pixel values Rec_(corrected)(x_(m), y_(m)) according topixels as results of compensating for the pixel values Rec(x_(m), y_(m))in the pixel groups {(x_(m), y_(m))}.

The video encoding apparatus 10 and the video decoding apparatus 20 mayclassify the restored pixels according to a predetermined standard, soas to determine the pixel groups having the pixel values to becompensated for.

The pixel groups may be classified according to extreme and/or edgevalue levels, according to an exemplary embodiment. A local extremeand/or edge value includes a local minimum value and a local maximumvalue. A local minimum value f(x_(min), y_(min)) and a local maximumvalue f(x_(max), y_(max)) at a neighboring coordinate (x,y) in apredetermined range ε are respectively defined in Equations 3 and 4below, with respect to a quadratic function f(x,y).

f(x,y)>f(x _(min) ,y _(min)), if |x _(min) −x|+|y _(min) −y|<ε andε>0.  Equation 3

f(x,y)<f(x _(max) ,y _(max)), if |x _(max) −x|+|y _(max) −y|<ε(wherein,ε>0).  Equation 4

Also, the local minimum value f(x_(min), y_(min)) and the local maximumvalue f(x_(max), y_(max)) may be respectively defined in Equations 5 and6 below, with respect to a discrete signal pixel (x,y).

f(x _(min) ,y _(min))<f(x _(min)+1,y _(min))

f(x _(min) ,y _(min))<f(x _(min)−1,y _(min))

f(x _(min) ,y _(min))<f(x _(min) ,y _(min)+1)

f(x _(min) ,y _(min))<f(x _(min) ,y _(min)−1).  Equation5

f(x _(max) ,y _(max))>f(x _(max)+1,y _(max))

f(x _(max) ,y _(ma) x)>f(x _(max)−1,y _(max))

f(x _(max) ,y _(max))>f(x _(max) ,y _(max)+1)

f(x _(max) ,y _(max))>f(x _(max) ,y _(max)−1).  Equation 6

The video encoding apparatus 10 and the video decoding apparatus 20 maydetermine pixels corresponding to the extreme and/or edge value fromamong predetermined neighboring restored pixels on horizontal andvertical lines, according to Equations 5 and 6. Also, the More neighborpixels, including, for example, pixels (x_(max)+1, y_(max)+1),(x_(max)−1, y_(max)+1), (x_(max)+1, y_(max)−1) and (x_(max)−1,y_(max)−1) on diagonal lines, may be included to pixel classificationprocess. Predetermined pixels may be excluded from pixel groups. Forexample, if only pixels on the same line may be classified into a pixelgroup, other pixels away from the corresponding line may be excludedfrom the pixel group.

A planarization phenomenon may be generated by a general video encodingand decoding system. Accordingly, a local minimum value in a restoredimage is higher than a pixel value of an original image, and an errorbetween local minimum values of the restored image and the originalimage is a positive value. Also, a local maximum value in the restoredimage is lower than a pixel value of the original image, and an errorbetween local maximum values of the restored image and the originalimage is a negative value.

Accordingly, the video encoding apparatus 10 and the video decodingapparatus 20 may determine average values of errors of local minimumvalues and local maximum values of a predetermined pixel group between arestored image and an original image, and compensate for pixel values ofrestored pixels in the predetermined pixel group. Hereinafter, a methodof determining an extreme and/or edge value level of restored pixels ofa predetermined pixel group, which is performed by the compensationvalue and pixel group determiner 16 of the video encoding apparatus 10,and the pixel group determiner 26 of the video decoding apparatus 20,will be described with reference to FIG. 3.

FIG. 3 illustrates neighboring restored pixels 32, 34, 35, and 37 to becompared to a current restored pixel 30 so as to determine an extremeand/or edge value level of the current restored pixel 30, according toan exemplary embodiment. For the convenience of explanation, FIG. 3illustrates only neighboring restored pixels 32, 34, 35, and 37.However, pixels compared to the current restored pixel 30 according toan exemplary embodiment do not limited to neighboring restored pixels32, 34, 35, and 37 on horizontal and vertical lines.

The compensation value and pixel group determiner 16 and the pixel groupdeterminer 26 may determine an extreme and/or edge value level of thecurrent restored pixel 30 by comparing the neighboring restored pixels32, 34, 35, and 37 that are disposed to the top, left, right, and bottomof the current restored pixel 30, respectively, with the currentrestored pixel 30. When a parameter ‘Pixel_Type’ denotes the extremeand/or edge value level of the current restored pixel 30, Rec[x][y]denotes a pixel value of the current restored pixel 30, and Rec[x][y−1],Rec[x−1][y], Rec[x+1][y], and Rec[x][y+1] respectively denote pixelvalues of the neighboring restored pixels, the extreme and/or edge valuelevel may be determined as following classification formulas:

Pixel_Type=0;

if (Rec[x][y]>Rec[x−1][y])Pixel_Type++;

if (Rec[x][y]<Rec[x−1][y])Pixel_Type−−;

if (Rec[x][y]>Rec[x+1][y])Pixel_Type++;

if (Rec[x][y]<Rec[x+1][y])Pixel_Type−−;

if (Rec[x][y]>Rec[x][y−1])Pixel_Type++;

if (Rec[x][y]<Rec[x][y−1])Pixel_Type−−;

if (Rec[x][y]>Rec[x][y+1])Pixel_Type++;

if (Rec[x][y]<Rec[x][y+1])Pixel_Type−−.

Maximum and minimum values of ‘Pixel_Type’ may be assumed as +4 and −4.

If pixel classification pattern has different shape from an exemplarypixel classification pattern shown on FIG. 3, then classificationformulas should be correspondingly modified. For example, during edgedetection on diagonal direction of angle 45°, pixels 31 and 38 arecompared to current pixel 30. The extreme and/or edge value level may bedetermined as following modified classification formulas:

Pixel_Type=0;

if (Rec[x][y]>Rec[x−1][y−1])Pixel_Type++;

if (Rec[x][y]<Rec[x−1][y−1])Pixel_Type−−;

if (Rec[x][y]>Rec[x+1][y+1])Pixel_Type++;

if (Rec[x][y]<Rec[x+1][y+1])Pixel_Type−−.

Correspondingly, maximum and minimum values of ‘Pixel_Type’ may beassumed as +2 and −2.

In other words, the extreme and/or edge value level is determined bycomparing pixel values of the neighboring restored pixels 32, 34, 35,and 37 in a predetermined range of the current restored pixel 30 with apixel value of the current restored pixel 30. When the extreme and/oredge value level of the current restored pixel 30 is a maximum extremeand/or edge value level, i.e., M, the current restored pixel 30 may bedetermined as a local maximum pixel, and when the extreme and/or edgevalue level of the current restored pixel 30 is a minimum extreme and/oredge value level, i.e., −M, the current restored pixel 30 may bedetermined as a local minimum pixel. A value of M may be determinedbased on a predetermined number of the analyzed neighboring restoredpixels. The compensation value and pixel group determiner 16 and thepixel group determiner 26 may determine restored pixels determined as alocal maximum pixel and a local minimum pixel as pixels to becompensated for.

As such, the compensation value and pixel group determiner 16 and thepixel group determiner 26 determine the extreme and/or edge value levelsof the restored pixels in a current data unit, and determine a pixelgroup including restored pixels having the extreme and/or edge valuelevel of M and a pixel group including restored pixels having theextreme and/or edge value level of −M. The compensation value and pixelgroup determiner 16 may determine the average value of the errors of thepixel values between the restored pixels and corresponding originalpixels according to pixel groups, and determine the compensation valuebased on the average value. The pixel group determiner 26 and therestored pixel compensator 28 may compensate for the pixel values of therestored pixels according to pixel groups, by using the compensationvalue extracted from the received information about the compensationvalue.

The compensation value and pixel group determiner 16 and the pixel groupdeterminer 26 may determine a pixel group including restored pixelsadjacent to the local maximum pixel and local minimum pixel as a targetto be compensated for. Accordingly, the compensation value and pixelgroup determiner 16 and the pixel group determiner 26 may determinecompensation values for extreme and/or edge value levels in apredetermined range including the maximum extreme and/or edge valuelevel and the minimum extreme and/or edge value level. For example,since the maximum extreme and/or edge value level is M as describedabove, restored pixels having an extreme and/or edge value level of M−1are adjacent to the local maximum pixel.

Accordingly, the compensation value and pixel group determiner 16 andthe pixel group determiner 26 may determine pixel groups includingrestored pixels having an extreme and/or edge value level higher than apredetermined positive value as pixel groups adjacent to the maximumextreme and/or edge value level, and pixel groups including restoredpixels having an extreme and/or edge value level lower than apredetermined negative value as pixel groups adjacent to the minimumextreme and/or edge value level. For example, when an extreme and/oredge value level is higher than m or lower than −m, i.e., when theextreme and/or edge value level is −M, −(M−1), −(M−2), . . . , −(m+1),(m+1), . . . , (M−1), or M, a compensation value according to extremeand/or edge value levels may be determined.

Alternatively, the compensation value and pixel group determiner 16 maycalculate the average value of the errors between the restored pixelsand the corresponding original pixels according to pixel groups adjacentto the maximum extreme and/or edge value level, and determine thecompensation value according to the pixel groups. Also, the pixel groupdeterminer 26 and the restored pixel compensator 28 may compensate forthe pixel values of the restored pixels according to pixel groups, byusing the compensation values according to pixel groups, which areextracted from the information about the compensation value.

Here, the 4 neighboring restored pixels 32, 34, 35, and 37 disposed tothe top, left, right, and bottom of the current restored pixel 30,respectively, are used to determine the extreme and/or edge value level,but in order to classify the extreme and/or edge value level in detail,8 restored pixels 31 through 38 around the current restored pixel 30 maybe used as neighboring restored pixels to determine the extreme and/oredge value level of the current restored pixel 30.

Alternatively, the video encoding apparatus 10 and the video decodingapparatus 20 may classify pixel values into a number of bands equal toor above a predetermined number.

For example, when a bit depth of restored pixels is N, a total range ofpixel values Rec[x][y] of the restored pixels is 0≦Rec[x][y]≦2^(N)−1. Inother words, a maximum value Max of the pixel value Rec[x][y] is2^(N)−1, and a section of the restored pixels is [0, Max]. Thecompensation value and pixel group determiner 16 and the pixel groupdeterminer 26 may split the section of the restored pixels into L bands.In other words, the band of the restored pixel may be split into [0,(Max+1)/L−1], [Max/L, 2*(Max+1)/L−1], and [2*Max/L, 3*(Max+1)/L−1]through [(L−1)*Max/L, L*(Max+1)/L−1].

Actual original data may be inside diapason [Min, Max]. A minimum valueMin and a maximum value Max are not mandatorily equal to 0 and 2^(N)−1,respectively. The number of different values may correspond to a rangeof Actual original data, that is, ‘Range=Max−Min+1’. If bands of therestored pixels are split uniformly, uniform bands are split into [Min,Range/L−1], [Max/L, 2*Range/L−1], and [2*Max/L, 3*Range/L−1] through[(L−1)*Range/L, Max]. In other exemplary embodiment, bands of therestored pixels may be non-uniformly split.

The number L of the split bands of the section [0, Max] of the restoredpixels may be a multiple of 2, and may be 16 or more for quickcalculation. Also, for quick calculation, the number L may be determinedin such a way that a number p of most significant bits of the restoredpixels is within an index of 2. For example, when the most significantbits of the restored pixel is 4 bits (p=4), and an extended bit depth ofthe restored pixels is 12 bits, the number L may be 2^(p)=16.Accordingly, the band of the restored pixels of the extended bit depthmay be split as shown in Table 1 below.

TABLE 1 Band No. 0 1 2 . . . 16 Band of Pixel [0, 255] [256, 511] [512,767] . . . [3840, Values of 4095] Restored Pixel Hexadecimal [0x0000,[0x0100, [0x0200, . . . [0x0F00, Expression of 0x00FF] 0x01FF] 0x02FF]0x0FFF] Pixel Value

Since bit calculation is easily performed when the band of the pixelvalues is split based on the number of most significant bits of therestored pixels, the pixel group determiner 26 may efficiently performcalculation for determining the band.

The compensation value and pixel group determiner 16 and the pixel groupdeterminer 26 may classify restored pixels in the same bands into apixel group according to bands. The bands may be split based on actualminimum and maximum values of original or reconstructed signal.

An average value of errors between the restored pixels included in thepixel group according to bands, and the original pixels is not 0.Accordingly, the compensation value and pixel group determiner 16 maydetermine a compensation value by using the average value according tobands. Also, the pixel group determiner 26 and the restored pixelcompensator 28 may compensate for pixel values of the restored pixels inthe pixel group according to bands, by using the compensation valuesaccording to bands.

Alternatively, the video encoding apparatus 10 and the video decodingapparatus 20 may classify restored pixels into pixel groups includingrestored pixels forming a predetermined line.

The compensation value and pixel group determiner 16 and the pixel groupdeterminer 26 may analyze image characteristics of the restored image,and detect lines in a vertical direction, horizontal direction, adiagonal direction, a curved direction and a boundary direction of apredetermined object. The compensation value and pixel group determiner16 and the pixel group determiner 26 may determine restored pixelsforming the same line as a pixel group according to lines.

An average value of errors of pixel values between restored pixelsincluded in the pixel group according to lines, and original pixels isalso not 0. The compensation value and pixel group determiner 16 maydetermine a compensation value by using the average value according tolines. The pixel group determiner 26 and the restored pixel compensator28 may compensate for pixel values of the restored pixels in the pixelgroup according to lines, by using the compensation value according tolines.

The compensation value and pixel group determiner 16 and the pixel groupdeterminer 26 may determine the compensation value according to extremeand/or edge value levels, according to data units, such as imagesequences, frames, or blocks of a video. The transmitter 18 may encodeand transmit the information related to the compensation value asoverhead information. Accuracy of the compensation value increases as adata unit for determining the compensation value according to extremeand/or edge value levels decreases, but overhead may increase sinceadditional information for encoding and transmitting the informationrelated to the compensation value may increase.

Also, the extractor 22 may extract the information related to thecompensation value from overhead information or slice header informationand compensate for the pixel values of the restored pixels by using thecompensation value.

The restored image generators 14 and 24 may selectively perform adaptiveloop filtering on image data decoded in a spatial domain. The restoredimage generators 14 and 24 may restore a current picture by continuouslyperforming one-dimensional filtering in a horizontal direction and in avertical direction, according to the adaptive loop filtering.

The transmitter 18 of the video encoding apparatus 10 may encode andoutput a filter coefficient used in the adaptive loop filtering. Also,since a type, a number, a size, a quantization bit, a coefficient, afiltering direction of each one-dimensional filter, and whetherfiltering and running filtering are performed may be set for theadaptive loop filtering, information about a set of one-dimensionalfilters of loop filtering may be encoded and transmitted.

The restored image generator 24 may induce a filter coefficient of eachone-dimensional filter by using residual information of the filtercoefficient extracted from the extractor 22.

For example, a current filter coefficient of each one-dimensional filtermay be induced by adding a difference between the current filtercoefficient and a previous filter coefficient to the previous filtercoefficient. The continuous one-dimensional filtering may be performedon deblocked data by using the induced filter coefficient of eachone-dimensional filter. The deblocking is performed to reduce a blockingeffect of decoded data, and the loop filtering minimizes an errorbetween the restored image and the original image.

For deeper understanding, the loop filtering using the continuousone-dimensional filtering in a horizontal direction and a verticaldirection will be described with reference to the following equations.

The current filter coefficient may be induced according to Equation 7below.

c[i][j]=adaptive_loop_filter_prev[i][j]+adaptive_loop_filter[i][j].  Equation7

Here, i denotes an index of a one-dimensional filter and j denotes anindex of a filter coefficient of a one-dimensional filter. c[i][j]denotes a current filter coefficient, adaptive_loop_filter_prev[i][j]denotes a previous filter coefficient, and adaptive_loop_filter[i][j]denotes a residual component of a filter coefficient transmitted asfilter coefficient information.

In other words, the current filter coefficient may be induced from a sumof the previous filter coefficient and the residual component. In orderto induce a following filter coefficient after inducing the currentfilter coefficient, the current filter coefficient c[i][j] is updated toadaptive_loop_filter_prev[i][j].

The loop filtering using continuous one-dimensional filtering may beperformed according to Equations 8 and 9 below. In Equations 8 and 9, idenotes an index in a width direction of a current picture and j denotesan index in a height direction of the current picture.

q _(i,j)=(p _(i,j−4) *c[0][4]+p _(i,j−3) *c[0][3]+p _(i,j−2) *c[0][2]+p_(i,j−1) *c[0][1]+p _(i,j) *c[0][0]+p _(i,j+1) *c[0][1]+p _(i,j+2)*c[0][2]+p _(i,j+3) *c[0][3]+p _(i,j+4) *c[0][4]).  Equation 8

Here, p_(i,j) denotes deblocked data of the current picture, and q_(i,j)denotes one-dimensional filtered data in a horizontal direction withrespect to the deblocked data. 5 filter coefficients are used tosymmetrically filter 9 pieces of deblocked data, by using a filtercoefficient c of a symmetrical filter.

f _(i,j)=(q _(i,j−4) *c[1][4]+q _(i,j−3) *c[1][3]+q _(i,j−2) *c[1][2]+q_(i,j−1) *c[1][1]+q _(i,j) *c[1][0]+q _(i,j+1) *c[1][1]+q _(i,j+2)*c[1][2]+q _(i,j+3) *c[1][3]+q _(i,j+4) *c[1][4]).  Equation 9

Here, f_(i,j) denotes one-dimensional filtered data in a verticaldirection with respect to the one-dimensional filtered data q_(i,j).Since the filter coefficient c uses a running filtering method,one-dimensional filtering is continuously performed in a verticaldirection on the one-dimensional filtered data in a horizontaldirection.

In the symmetrical filter, a one-dimensional filter is able to setcoefficients of all filters by only using a small amount ofcoefficients, compared to a two-dimensional filter. Accordingly, bitsrelated to filter characteristics of a plurality of one-dimensionalfilters, which are inserted into a transmission bitstream, may berelatively low compared to a two-dimensional filter.

Also, memory capacity for storing temporary data during filtering issmaller in the one-dimensional filter than the two-dimensional filter.Filtering throughput of the two-dimensional filter is remarkably largecompared to that of the one-dimensional filtering. In the runningfiltering, it is not possible to perform a parallel process according tomultiple filtering by using the two-dimensional filter, but it ispossible to perform a parallel process by using the one-dimensionalfilter.

However, the loop filtering is not limited to the continuousone-dimensional filtering in horizontal and vertical directions. Theloop filtering may be performed as a predetermined number ofone-dimensional filters perform continuous one-dimensional filtering,wherein each one-dimensional filtering is performed in a predetermineddirection.

The video decoding apparatus 20 may receive information about a set ofone-dimensional filters, aside from the information about the filtercoefficient, so as to check a type, a number, a size, a quantizationbit, a coefficient, a filtering direction of each one-dimensionalfilter, and whether filtering and running filtering are performed.Accordingly, the restored image generator 24 may perform loop filteringby combining various one-dimensional filters.

The adaptive loop filtering performed by the restored image generators14 and 24 will now be described with reference to FIGS. 4 and 5.

FIG. 4 is a flowchart for describing adaptive loop filtering accordingto an exemplary embodiment.

Loop filtering may be performed as a plurality of one-dimensionalfilters continuously performing filtering. In operation 41, decodedimage data is received. Alternatively, image data on which deblockingfiltering is performed after decoding may be received. In operation 42,it is determined whether all of first through Nth filters are to beused. If it is determined that the first through Nth filters are not tobe used, operation 46 is performed. If it is determined that the firstthrough Nth filters are to be used in operation 42, one-dimensionalfiltering may be performed according to a filtering order, for example,the first filter performs one-dimensional filtering in a first filteringdirection in operation 43 and the second filter performs one-dimensionalfiltering in a second filtering direction in operation 44, until the Nthfilter performs one-dimensional filtering in an Nth filtering directionin operation 45.

In operation 46, decoded image data, deblocked image data, orcontinuously one-dimensional filtered data is stored in a buffer orreproduced by a reproducer.

A filtering direction of a one-dimensional filter may be adaptivelydetermined according to characteristics of a local image, by analyzingthe characteristics. For example, the filtering direction may beadaptively determined as an edge direction of a local image so as toconserve an edge of the local image.

FIG. 5 is a flowchart for describing adaptive loop filtering accordingto another exemplary embodiment.

When decoded image data or deblocked image data is received in operation51, an edge is detected for each pixel of the decoded image data ordeblocked image data in operation 52. In operation 53, one-dimensionalfiltering is performed according to the detected edge, and filtered datais stored or reproduced by a reproducer in operation 54.

Information about a set of one-dimensional filters including a filteringdirection determined according to the edge is encoded and provided to adecoder while encoding a video. Information about a loop filter is readfrom received data while decoding a video, and one-dimensional filteringaccording to a filtering direction, such as an edge direction, may beperformed by a predetermined one-dimensional filter.

A post-process constituting the loop filtering may reduce distortionbetween an original image and a restored image, which is generated dueto complex loss compression. Also, a loop filtered image may be used asa reference image so as to improve quality of an image obtained byperforming prediction or motion compensation.

Accordingly, the restored image generators 14 and 24 may selectivelyperform adaptive loop filtering considering characteristics of an image,a system environment, or user requirements by combining one-dimensionalfilters having various characteristics. Since continuous one-dimensionalfilters are used instead of a two-dimensional filter so as to performthe adaptive loop filtering, the adaptive loop filtering may beadvantageous in terms of memory, throughput, amount of transmissionbits, etc., compared to the two-dimensional filter. When the restoredimage generators 14 and 24 perform the adaptive loop filtering, thetransmitter 18 and the extractor 22 transmit and receive informationobtained by encoding a residual component of an encoded filtercoefficient, and thus an amount of information used for the adaptiveloop filtering may be reduced.

FIG. 6 is a flowchart illustrating a method of encoding a video forcompensating for a pixel value, according to an exemplary embodiment.

In operation 62, an input image sequence is encoded. In operation 64,the encoded image data is decoded, and a restored image is generated byperforming loop filtering on the decoded image data. The restored imagemay be generated by performing adaptive loop filtering, wherein at leastone one-dimensional filtering operation is continuously performed on thedecoded image data or deblocked image data.

In operation 66, a compensation value about an error between eachrestored pixel of a predetermined group in the restored image, and acorresponding original image, and a pixel group including restoredpixels to be compensated for are determined. The pixel group includingthe restored pixels to be compensated for may be determined according toextreme and/or edge value levels of pixel values, bands of pixel values,or lines. The compensation value according to pixel groups may bedetermined based on an average value of the errors.

In operation 68, the compensation value is encoded, and a bitstream ofthe encoded compensation value and the encoded input image sequence istransmitted. When the compensation value is determined according to moredetailed pixel groups, the pixel values may be accurately compensatedfor, but overhead may increase.

FIG. 7 is a flowchart illustrating a method of decoding a video forcompensating for a pixel value, according to an exemplary embodiment.

In operation 72, a bitstream about an encoded image is received andparsed, and encoded image data and a compensation value are extractedfrom the bitstream.

In operation 74, the encoded image data is decoded, and a restored imageis generated by performing loop filtering on the decoded image data. Therestored image may be generated by performing adaptive loop filtering,wherein at least one one-dimensional filtering operation is continuouslyperformed on the decoded image data or deblocked image data.

In operation 76, a pixel group including restored pixels to becompensated for by using the compensation value is determined from amongrestored pixels in the restored image. The pixel group including therestored pixels to be compensated for by using the compensation valuemay be determined according to extreme and/or edge value levels of pixelvalues of the restored pixels, bands of the pixel values, or lines,according to a method of determining a pixel group, based on informationrelated to the compensation value. In operation 78, the restored imagehaving a compensated error may be output by compensating for an errorbetween restored pixels of the determined pixel group and originalpixels by using the compensation value.

According to the method of encoding a video and the method of decoding avideo, the quality of the restored image may be improved by compensatingfor a systematic error of the restored image, and the amount oftransmission bits of additional information for improving the quality ofthe restored image may be reduced since only the information about thecompensation value according to pixel groups is encoded and transmitted,and information about locations of restored pixels to be compensated foris not transmitted.

Hereinafter, the encoding and decoding a video for compensating for apixel value after performing loop filtering based on coding units havinga tree structure, according to exemplary embodiments, will be describedwith reference to FIGS. 8 through 22.

FIG. 8 is a block diagram of a video encoding apparatus 80 for encodinga video for compensating for a pixel value after performing loopfiltering based on coding units having a tree structure, according to anexemplary embodiment.

The video encoding apparatus 80 according to the current exemplaryembodiment includes an encoder 81, a restored image generator 84, acompensation value and pixel group determiner 87, and a transmitter 88.The encoder 81 includes a maximum coding unit splitter 82 and codeddepth and encoding mode determiner 83. The restored image generator 84includes a decoder 85 and a loop filtering performer 86.

The encoder 81 encodes an input image sequence. The encoder 81 mayencode the input image sequence based on coding units having a treestructure. The maximum coding unit splitter 82 may split a currentpicture based on a maximum coding unit for the current picture of animage. The maximum coding unit according to an exemplary embodiment maybe a data unit having a size of 32×32, 64×64, 128×128, 256×256, etc.,wherein a shape of the data unit is a square having a width and lengthin squares of 2.

If the current picture is larger than the maximum coding unit, imagedata of the current picture may be split into the at least one maximumcoding unit. The image data may be output to the coded depth andencoding mode determiner 83 according to the at least one maximum codingunit.

A coding unit according to an exemplary embodiment may be characterizedby a maximum size and a depth. The depth denotes a number of times thecoding unit is spatially split from the maximum coding unit, and as thedepth deepens, deeper encoding units according to depths may be splitfrom the maximum coding unit to a minimum coding unit. A depth of themaximum coding unit is an uppermost depth and a depth of the minimumcoding unit is a lowermost depth. Since a size of a coding unitcorresponding to each depth decreases as the depth of the maximum codingunit deepens, a coding unit corresponding to an upper depth may includea plurality of coding units corresponding to lower depths.

As described above, the image data of the current picture is split intothe maximum coding units according to a maximum size of the coding unit,and each of the maximum coding units may include deeper coding unitsthat are split according to depths. Since the maximum coding unitaccording to an exemplary embodiment is split according to depths, theimage data of a spatial domain included in the maximum coding unit maybe hierarchically classified according to depths.

A maximum depth and a maximum size of a coding unit, which limit thetotal number of times a height and a width of the maximum coding unitare hierarchically split, may be predetermined.

The coded depth and encoding mode determiner 83 encodes at least onesplit region obtained by splitting a region of the maximum coding unitaccording to depths, and determines a depth to output a finally encodedimage data according to the at least one split region. In other words,the coded depth and encoding mode determiner 83 determines a coded depthby encoding the image data in the deeper coding units according todepths, according to the maximum coding unit of the current picture, andselecting a depth having the least encoding error. Thus, the encodedimage data of the coding unit corresponding to the determined codeddepth is output. Also, the coding units corresponding to the coded depthmay be regarded as encoded coding units. The determined coded depth andthe encoded image data according to the determined coded depth areoutput to the transmitter 88.

The image data in the maximum coding unit is encoded based on the deepercoding units corresponding to at least one depth equal to or below themaximum depth, and results of encoding the image data are compared basedon each of the deeper coding units. A depth having the least encodingerror may be selected after comparing encoding errors of the deepercoding units. At least one coded depth may be selected for each maximumcoding unit.

The size of the maximum coding unit is split as a coding unit ishierarchically split according to depths, and as the number of codingunits increases. Also, even if coding units correspond to the same depthin one maximum coding unit, it is determined whether to split each ofthe coding units corresponding to the same depth to a lower depth bymeasuring an encoding error of the image data of each of the codingunits, separately. Accordingly, even when image data is included in onemaximum coding unit, the image data is split to regions according to thedepths and the encoding errors may differ according to regions in theone maximum coding unit, and thus the coded depths may differ accordingto regions in the image data. Thus, one or more coded depths may bedetermined in one maximum coding unit, and the image data of the maximumcoding unit may be split according to coding units of at least one codeddepth.

Accordingly, the coded depth and encoding mode determiner 83 maydetermine coding units having a tree structure included in the maximumcoding unit. The ‘coding units having a tree structure’ according to anexemplary embodiment include coding units corresponding to a depthdetermined to be the coded depth, from among all deeper coding unitsincluded in the maximum coding unit. A coding unit of a coded depth maybe hierarchically determined according to depths in the same region ofthe maximum coding unit, and may be independently determined indifferent regions. Similarly, a coded depth in a current region may beindependently determined from a coded depth in another region.

A maximum depth according to an exemplary embodiment is an index relatedto the number of splitting times from a maximum coding unit to a minimumcoding unit. A maximum depth according to an exemplary embodiment maydenote the total number of splitting times from the maximum coding unitto the minimum coding unit. For example, when a depth of the maximumcoding unit is 0, a depth of a coding unit, in which the maximum codingunit is split once, may be set to 1, and a depth of a coding unit, inwhich the maximum coding unit is split twice, may be set to 2. Here, ifthe minimum coding unit is a coding unit in which the maximum codingunit is split four times, 5 depth levels of depths 0, 1, 2, 3 and 4exist, and thus the maximum depth may be set to 4.

Prediction encoding and transformation may be performed according to themaximum coding unit. The prediction encoding and the transformation arealso performed based on the deeper coding units according to a depthequal to or depths less than the maximum depth, according to the maximumcoding unit. Transformation may be performed according to method oforthogonal transformation or integer transformation.

Since the number of deeper coding units increases whenever the maximumcoding unit is split according to depths, encoding including predictionencoding and transformation is performed on all of the deeper codingunits generated as the depth deepens. For convenience of description,the prediction encoding and the transformation will now be describedbased on a coding unit of a current depth, in a maximum coding unit.

The video encoding apparatus 80 may variously select a size or shape ofa data unit for encoding the image data. In order to encode the imagedata, operations, such as prediction encoding, transformation, andentropy encoding, are performed, and at this time, the same data unitmay be used for all operations or different data units may be used foreach operation.

For example, the video encoding apparatus 80 may select not only acoding unit for encoding the image data, but also a data unit differentfrom the coding unit so as to perform the prediction encoding on theimage data in the coding unit.

In order to perform prediction encoding in the maximum coding unit, theprediction encoding may be performed based on a coding unitcorresponding to a coded depth, i.e., based on a coding unit that is nolonger split to coding units corresponding to a lower depth.Hereinafter, the coding unit that is no longer split and becomes a basisunit for prediction encoding will now be referred to as a ‘predictionunit’. A partition obtained by splitting the prediction unit may includea prediction unit or a data unit obtained by splitting at least one of aheight and a width of the prediction unit.

For example, when a coding unit of 2N×2N (where N is a positive integer)is no longer split and becomes a prediction unit of 2N×2N, a size of apartition may be 2N×2N, 2N×N, N×2N, or N×N. Examples of a partition typeinclude symmetrical partitions that are obtained by symmetricallysplitting a height or a width of the prediction unit, partitionsobtained by asymmetrically splitting the height or the width of theprediction unit, such as 1:n or n:1, partitions that are obtained bygeometrically splitting the prediction unit, and partitions havingarbitrary shapes.

A prediction mode of the prediction unit may be at least one of an intramode, an inter mode, and a skip mode. For example, the intra mode or theinter mode may be performed on the partition of 2N×2N, 2N×N, N×2N, orN×N. Also, the skip mode may be performed only on the partition of2N×2N. The encoding is independently performed on one prediction unit ina coding unit, thereby selecting a prediction mode having a leastencoding error.

The video encoding apparatus 80 may also perform the transformation onthe image data in a coding unit based not only on the coding unit forencoding the image data, but also based on a data unit that is differentfrom the coding unit.

In order to perform the transformation in the coding unit, thetransformation may be performed based on a data unit having a sizesmaller than or equal to the coding unit. For example, the data unit forthe transformation may include a data unit for an intra mode and a dataunit for an inter mode.

A data unit used as a base of the transformation will now be referred toas a ‘transformation unit’. A transformation depth indicating the numberof splitting times to reach the transformation unit by splitting theheight and width of the coding unit may also be set in thetransformation unit. For example, in a current coding unit of 2N×2N, atransformation depth may be 0 when the size of a transformation unit isalso 2N×2N, may be 1 when each of the height and width of the currentcoding unit is split into two equal parts, totally split into 4̂1transformation units, and the size of the transformation unit is thusN×N, and may be 2 when each of the height and width of the currentcoding unit is split into four equal parts, totally split into 4̂2transformation units and the size of the transformation unit is thusN/2×N/2. For example, the transformation unit may be set according to ahierarchical tree structure, in which a transformation unit of an uppertransformation depth is split into four transformation units of a lowertransformation depth according to the hierarchical characteristics of atransformation depth.

Similar to the coding unit, the transformation unit in the coding unitmay be recursively split into smaller sized regions, so that thetransformation unit may be determined independently in units of regions.Thus, residual data in the coding unit may be split according to thetransformation having the tree structure according to transformationdepths.

Encoding information according to coding units corresponding to a codeddepth uses not only information about the coded depth, but also aboutinformation related to prediction encoding and transformation.Accordingly, the coded depth and encoding mode determiner 83 not onlydetermines a coded depth having a least encoding error, but alsodetermines a partition type in a prediction unit, a prediction modeaccording to prediction units, and a size of a transformation unit fortransformation.

The coded depth and encoding mode determiner 83 may measure an encodingerror of deeper coding units according to depths by usingRate-Distortion Optimization based on Lagrangian multipliers.

The restored image generator 84 decodes encoded image data and generatesa restored image by performing loop filtering on the decoded image data.The decoder 85 included in the restored image generator 84 decodes imagedata based on coding units having a tree structure, which is encoded bythe encoder 81. The decoder 85 may decode the encoded image data andoutput image data of a spatial domain according to maximum coding units,based on the coded depth and the encoding mode determined by the codeddepth and encoding mode determiner 83.

The loop filtering performer 86 included in the restored image generator84 may perform in-loop filtering on the decoded image data. The sameadaptive loop filtering selectively performed by the restored imagegenerator 14 may be performed by the loop filtering performer 86.Accordingly, the loop filtering performer 86 may continuously performone-dimensional filtering in a horizontal direction and one-dimensionalfiltering in a vertical direction to restore the current picture. Theloop filtering performer 86 may output the restored image to thecompensation value and pixel group determiner 87.

The compensation value and pixel group determiner 87 determines acompensation value about an error between each of the restored pixels ofa predetermined group in the restored image, and a correspondingoriginal pixel, and a group pixel including restored pixels having pixelvalues to be compensated for. The compensation value and pixel groupdeterminer 87 is a technical element corresponding to the compensationvalue and pixel group determiner 16.

Accordingly, the compensation value and pixel group determiner 87 maydetermine an extreme and/or edge value level of neighboring restoredpixels of the restored image according to restored pixels, and classifythe neighboring restored pixels into pixel groups according to extremeand/or edge value levels. Alternatively, the compensation value andpixel group determiner 87 may classify the restored pixels into pixelgroups according to bands based on a pixel value. Alternatively, thecompensation value and pixel group determiner 87 may detect lines in apredetermined direction by analyzing the restored image, and classifythe restored pixels into pixel groups according to lines, which includethe restored pixels on the same line.

The compensation value and pixel group determiner 87 may individuallydetermine a compensation value for each pixel group by using an averagevalue of the errors between the restored pixel and the correspondingoriginal pixel. The compensation value and pixel group determiner 87 maydetermine a restored pixel to be compensated for according to at leastone data unit from among an image sequence, a slice, a frame, and acoding unit of an input video, and determine a compensation valuecorresponding to the determined restored pixel to be compensated for.Information about the compensation value and the pixel group determinedby the compensation value and pixel group determiner 87 may be output tothe transmitter 88.

The transmitter 88 outputs the image data of the maximum coding unit,which is encoded based on the at least one coded depth determined by thecoded depth and encoding mode determiner 83, and information, about theencoding mode according to the coded depth, in bitstreams. The imagedata encoded by the encoder 81 may be converted to a bitstream formatvia entropy encoding, and then inserted into a bitstream fortransmission.

Alternatively, the transmitter 88 may encode and insert the compensationvalue determined by the compensation value and pixel group determiner 86into the bitstream for transmission. Alternatively, the transmitter 88may receive additional information about a method of determining a pixelgroup from the compensation value and pixel group determiner 87, andencode and insert the additional information into a bitstream.

The encoded image data may be obtained by encoding residual data of animage.

The information about the encoding mode according to coded depth mayinclude information about the coded depth, about the partition type inthe prediction unit, the prediction mode, and the size of thetransformation unit.

The information about the coded depth may be defined by using splitinformation according to depths, which indicates whether encoding isperformed on coding units of a lower depth instead of a current depth.If the current depth of the current coding unit is the coded depth,image data in the current coding unit is encoded and output, and thusthe split information may be defined not to split the current codingunit to a lower depth. Alternatively, if the current depth of thecurrent coding unit is not the coded depth, the encoding is performed onthe coding unit of the lower depth, and thus the split information maybe defined to split the current coding unit to obtain the coding unitsof the lower depth.

If the current depth is not the coded depth, encoding is performed onthe coding unit that is split into the coding unit of the lower depth.Since at least one coding unit of the lower depth exists in one codingunit of the current depth, the encoding is repeatedly performed on eachcoding unit of the lower depth, and thus the encoding may be recursivelyperformed for the coding units having the same depth.

Since the coding units having a tree structure are determined for onemaximum coding unit, and information about at least one encoding mode isdetermined for a coding unit of a coded depth, information about atleast one encoding mode may be determined for one maximum coding unit.Also, a coded depth of the image data of the maximum coding unit may bedifferent according to locations since the image data is hierarchicallysplit according to depths, and thus information about the coded depthand the encoding mode may be set for the image data.

Accordingly, the transmitter 88 may assign encoding information about acorresponding coded depth and an encoding mode to at least one of thecoding unit; the prediction unit, and a minimum unit included in themaximum coding unit.

The minimum unit according to an exemplary embodiment is a rectangulardata unit obtained by splitting the minimum coding unit constituting thelower'most depth by 4. Alternatively, the minimum unit may be a maximumrectangular data unit that may be included in all of the coding units,prediction units, partition units, and transformation units included inthe maximum coding unit.

For example, the encoding information output through the transmitter 88may be classified into encoding information according to coding units,and encoding information according to prediction units. The encodinginformation according to the coding units may include the informationabout the prediction mode and about the size of the partitions. Theencoding information according to the prediction units may includeinformation about an estimated direction of an inter mode, about areference image index of the inter mode, about a motion vector, about achroma component of an intra mode, and about an interpolation method ofthe intra mode. Also, information about a maximum size of the codingunit defined according to pictures, slices, or GOPs, and informationabout a maximum depth may be inserted into a Sequence Parameter Set(SPS) or a header of a bitstream.

The transmitter 88 may encode and output a filter coefficient used inadaptive loop filtering. Also, since a type, a number, a size, aquantization bit, a coefficient, a filtering direction of eachone-dimensional filter, and whether filtering and running filtering areperformed may be set for the adaptive loop filtering, information abouta set of one-dimensional filters of loop filtering may be encoded andtransmitted.

In the video encoding apparatus 80, the deeper coding unit may be acoding unit obtained by dividing a height or a width of a coding unit ofan upper depth, which is one layer above, by two. In other words, whenthe size of the coding unit of the current depth is 2N×2N, the size ofthe coding unit of the lower depth is N×N. Also, the coding unit of thecurrent depth having the size of 2N×2N may include maximum 4 of thecoding unit of the lower depth.

Accordingly, the video encoding apparatus 80 may form the coding unitshaving the tree structure by determining coding units having an optimumshape and an optimum size for each maximum coding unit, based on thesize of the maximum coding unit and the maximum depth determinedconsidering characteristics of the current picture. Also, since encodingmay be performed on each maximum coding unit by using any one of variousprediction modes and transformations, an optimum encoding mode may bedetermined considering characteristics of the coding unit of variousimage sizes.

Thus, if an image having high resolution or a large data amount isencoded in a related art macroblock, a number of macroblocks per pictureexcessively increases. Accordingly, a number of pieces of compressedinformation generated for each macroblock increases, and thus it isdifficult to transmit the compressed information and data compressionefficiency decreases. However, by using the video encoding apparatus 80according to an exemplary embodiment, image compression efficiency maybe increased since a coding unit is adjusted while consideringcharacteristics of an image while increasing a maximum size of a codingunit while considering a size of the image.

Also, the amount of transmission bits of additional information may bereduced since information about a compensation value for compensatingfor a pixel value between a restored image and an original image, whichis required to improve the quality of the restored image by a decoder,is encoded and transmitted without information about a pixel location.

FIG. 9 is a block diagram of a video decoding apparatus 90 forcompensating for a pixel value after performing loop filtering based oncoding units having a tree structure, according to an exemplaryembodiment.

The video decoding apparatus 90 includes an extractor 91, a restoredimage generator 94, a pixel group determiner 97, and a restored pixelcompensator 98. The extractor 91 includes a receiver 92, and an imagedata, encoding mode information, loop filter coefficient information,and compensation value information extractor (hereinafter, referred toas an information extractor) 93. The restored image generator 94includes a decoder 95 and a loop filtering performer 96.

Definitions of terms such as a coding unit, a depth, a prediction unit,a transformation unit, and various encoding modes for various processesused to describe the video decoding apparatus 90 are identical to thosedescribed with reference to the video encoding apparatus 80 of FIG. 8.

The extractor 91 receives and parses a bitstream of an encoded image,and extracts encoded image data and a compensation value from thebitstream. The receiver 92 of the extractor 91 receives and parses thebitstream of the encoded image. The information extractor extracts theimage data according to maximum coding units from the parsed bitstream,and outputs the extracted image data to the decoder 95. The informationextractor 93 may extract information about a maximum size of a codingunit of a current picture, from a header about the current picture.

Also, the information extractor 93 extracts information about a codeddepth and an encoding mode for the coding units having a tree structureaccording to each maximum coding unit, from the parsed bitstream. Theextracted information about the coded depth and the encoding mode isoutput to the decoder 95. In other words, the image data in a bit stringis split into the maximum coding unit so that the decoder 95 decodes theimage data for each maximum coding unit.

The information about the coded depth and the encoding mode according tothe maximum coding unit may be set for information about at least onecoding unit corresponding to the coded depth, and information about anencoding mode may include information about a partition type of acorresponding coding unit corresponding to the coded depth, about aprediction mode, and a size of a transformation unit.

The information about the coded depth and the encoding mode according toeach maximum coding unit extracted by the information extractor 93 isinformation about a coded depth and an encoding mode determined togenerate a minimum encoding error when an encoder, such as the videoencoding apparatus 80, repeatedly performs encoding for each deepercoding unit according to depths according to each maximum coding unit.Accordingly, the video decoding apparatus 90 may restore an image bydecoding the image data according to a coded depth and an encoding modethat generates the minimum encoding error.

Since encoding information about the coded depth and the encoding modemay be assigned to a predetermined data unit from among a correspondingcoding unit, a prediction unit, and a minimum unit, the informationextractor 93 may extract the information about the coded depth and theencoding mode according to the predetermined data units. Thepredetermined data units to which the same information about the codeddepth and the encoding mode is assigned may be inferred to be the dataunits included in the same maximum coding unit.

The decoder 95 restores the current picture by decoding the image datain each maximum coding unit based on the information about the codeddepth and the encoding mode according to the maximum coding units. Inother words, the decoder 95 may decode the encoded image data based onthe extracted information about the partition type, the prediction mode,and the transformation unit for each coding unit from among the codingunits having the tree structure included in each maximum coding unit. Adecoding process may include a prediction including intra prediction andmotion compensation, and an inverse transformation. Inversetransformation may be performed according to a method of inverseorthogonal transformation or inverse integer transformation.

Also, the decoder 95 may perform inverse transformation according toeach transformation unit in the coding unit by reading thetransformation units having a tree structure, based on the informationabout the size of the transformation unit of the coding unit accordingto coded depths, so as to perform the inverse transformation accordingto maximum coding units.

The decoder 95 may determine at least one coded depth of a currentmaximum coding unit by using split information according to depths. Ifthe split information indicates that image data is no longer split inthe current depth, the current depth is a coded depth. Accordingly, thedecoder 95 may decode encoded data of at least one coding unitcorresponding to the each coded depth in the current maximum coding unitby using the information about the partition type of the predictionunit, the prediction mode, and the size of the transformation unit foreach coding unit corresponding to the coded depth, and output the imagedata of the current maximum coding unit.

In other words, data units containing the encoding information includingthe same split information may be gathered by observing the encodinginformation set assigned for the predetermined data unit from among thecoding unit, the prediction unit, and the minimum unit, and the gathereddata units may be considered to be one data unit to be decoded by thedecoder 95 in the same encoding mode.

When the information about the filter coefficient for the adaptive loopfiltering is inserted into the bitstream, the information extractor 93may extract the information about the filter coefficient from thebitstream. The loop filtering performer 96 may receive the informationabout the filter coefficient extracted by the information extractor 93,and generate the restored image by performing loop filtering on theimage data decoded by the decoder 95.

The same technical element of the restored image generator 24 may beapplied to the loop filtering performer 96. Accordingly, the loopfiltering performer 96 may selectively perform deblocking filtering andadaptive loop filtering on the decoded image data. The adaptive loopfiltering may be performed by using a continuous plurality ofone-dimensional filters.

The restored image generator 94 may induce a filter coefficient of eachone-dimensional filter by using residual information of the filtercoefficient extracted from the information extractor 93. For example, acurrent filter coefficient of each one-dimensional filter may be inducedby adding a difference between the current filter coefficient and aprevious filter coefficient to the previous filter coefficient. Thecontinuous one-dimensional filtering may be performed on deblocked databy using the induced filter coefficient of each one-dimensional filter.The deblocking is performed to reduce a blocking effect of decoded data,and the loop filtering minimizes an error between the restored image andthe original image.

The information extractor 93 extracts encoded image data and informationrelated to a compensation value from the bitstream. The informationrelated to the compensation value may include information about thecompensation value. Alternatively, if the information related to thecompensation value includes information about a method of determining apixel group to be compensated for by using the compensation value, theinformation extractor 93 may extract the compensation value and theinformation about the method of determining the pixel group to becompensated for from the bitstream. The information extractor 93 mayextract the compensation value or the information related to thecompensation value according to at least one data unit from among animage sequence, a slice, a frame, and a coding unit of an input video.

The pixel group determiner 97 may determine a pixel group including therestored pixel to be compensated for by using the compensation value,with respect to the restored pixels of a predetermined group in therestored image, by receiving the restored image generated by therestored image generator 94 and the compensation value extracted by theinformation extractor 93. The restored pixel compensator 98 compensatesfor the pixel value of the restored pixel by using the compensationvalue, and outputs the restored image having the restored pixel value byreceiving the compensation value extracted by the information extractor93 and information about the pixel group determined by the pixel groupdeterminer 97.

When the information about the method of determining the pixel group tobe compensated for is extracted by the information extractor 93, thepixel group determiner 97 may selectively determine the pixel grouphaving the pixel value to be compensated for based on the method. Forexample, the pixel group determiner 97 may determine whether to classifythe restored pixels according to the extreme and/or edge value levels,the bands of pixel values, or the lines, and determine the pixel grouphaving the pixel values to be compensated for, based on the method.Here, the restored pixel compensator 98 may compensate for the pixelvalues of the restored pixels in the pixel group by using thecompensation values for the pixel group according to extreme and/or edgevalue levels, bands of pixel values, or lines.

The video decoding apparatus 90 may obtain information about at leastone coding unit that generates the minimum encoding error when encodingis recursively performed for each maximum coding unit, and may use theinformation to decode the current picture. In other words, the codingunits having the tree structure determined to be the optimum codingunits in each maximum coding unit may be decoded. Also, the maximum sizeof coding unit is determined considering resolution and the amount ofimage data.

Accordingly, even if image data has high resolution and a large amountof data, the image data may be efficiently decoded and restored by usinga size of a coding unit and an encoding mode, which are adaptivelydetermined according to characteristics of the image data, by usinginformation about an optimum encoding mode received from an encoder.

The video encoding apparatus 80 and the video decoding apparatus 90 maycompensate for a systematic error generated between the restored imageand the original image when the encoded image is decoded and restored.

Encoding and decoding of a video based on coding units having a treestructure, according to exemplary embodiments.

FIG. 10 is a diagram for describing a concept of coding units accordingto an exemplary embodiment.

A size of a coding unit may be expressed in width×height, and may be64×64, 32×32, 16×16, and 8×8. A coding unit of 64×64 may be split intopartitions of 64×64, 64×32, 32×64, or 32×32, and a coding unit of 32×32may be split into partitions of 32×32, 32×16, 16×32, or 16×16, a codingunit of 16×16 may be split into partitions of 16×16, 16×8, 8×16, or 8×8,and a coding unit of 8×8 may be split into partitions of 8×8, 8×4, 4×8,or 4×4.

In video data 310, a resolution is 1920×1080, a maximum size of a codingunit is 64, and a maximum depth is 2. In video data 320, a resolution is1920×1080, a maximum size of a coding unit is 64, and a maximum depth is3. In video data 330, a resolution is 352×288, a maximum size of acoding unit is 16, and a maximum depth is 1. The maximum depth shown inFIG. 10 denotes a total number of splits from a maximum coding unit to aminimum decoding unit.

If a resolution is high or a data amount is large, a maximum size of acoding unit may be large so as to not only increase encoding efficiencybut also to accurately reflect characteristics of an image. Accordingly,the maximum size of the coding unit of the video data 310 and 320 havingthe higher resolution than the video data 330 may be 64.

Since the maximum depth of the video data 310 is 2, coding units 315 ofthe vide data 310 may include a maximum coding unit having a long axissize of 64, and coding units having long axis sizes of 32 and 16 sincedepths are deepened to two layers by splitting the maximum coding unittwice. Meanwhile, since the maximum depth of the video data 330 is 1,coding units 335 of the video data 330 may include a maximum coding unithaving a long axis size of 16, and coding units having a long axis sizeof 8 since depths are deepened to one layer by splitting the maximumcoding unit once.

Since the maximum depth of the video data 320 is 3, coding units 325 ofthe video data 320 may include a maximum coding unit having a long axissize of 64, and coding units having long axis sizes of 32, 16, and 8since the depths are deepened to 3 layers by splitting the maximumcoding unit three times. As a depth deepens, detailed information may beprecisely expressed.

FIG. 11 is a block diagram of an image encoder 400 based on codingunits, according to an exemplary embodiment.

The image encoder 400 performs operations of the encoder 81 of the videoencoding apparatus 80 to encode image data. In other words, an intrapredictor 410 performs intra prediction on coding units in an intramode, from among a current frame 405, and a motion estimator 420 and amotion compensator 425 performs inter estimation and motion compensationon coding units in an inter mode from among the current frame 405 byusing the current frame 405 and a reference frame 495.

Data output from the intra predictor 410, the motion estimator 420, andthe motion compensator 425 is output as a quantized transformationcoefficient through a transformer 430 and a quantizer 440. The quantizedtransformation coefficient is restored as data in a spatial domainthrough an inverse quantizer 460 and an inverse transformer 470, and therestored data in the spatial domain is output as the reference frame 495after being post-processed through a deblocking unit 480 and a loopfiltering unit 490. The quantized transformation coefficient may beoutput as a bitstream 455 through an entropy encoder 450.

In order for the image encoder 400 to be applied in the video encodingapparatus 80, elements of the image encoder 400, i.e., the intrapredictor 410, the motion estimator 420, the motion compensator 425, thetransformer 430, the quantizer 440, the entropy encoder 450, the inversequantizer 460, the inverse transformer 470, the deblocking unit 480, andthe loop filtering unit 490, perform operations based on each codingunit from among coding units having a tree structure while consideringthe maximum depth of each maximum coding unit.

Specifically, the intra predictor 410, the motion estimator 420, and themotion compensator 425 determine partitions and a prediction mode ofeach coding unit from among the coding units having a tree structurewhile considering the maximum size and the maximum depth of a currentmaximum coding unit, and the transformer 430 determines the size of thetransformation unit in each coding unit from among the coding unitshaving a tree structure.

FIG. 12 is a block diagram of an image decoder 500 based on codingunits, according to an exemplary embodiment.

A parser 510 parses encoded image data to be decoded and informationabout encoding used for decoding from a bitstream 505. The encoded imagedata is output as inverse quantized data through an entropy decoder 520and an inverse quantizer 530, and the inverse quantized data is restoredto image data in a spatial domain through an inverse transformer 540.

An intra predictor 550 performs intra prediction on coding units in anintra mode with respect to the image data in the spatial domain, and amotion compensator 560 performs motion compensation on coding units inan inter mode by using a reference frame 585.

The image data in the spatial domain, which passed through the intrapredictor 550 and the motion compensator 560, may be output as arestored frame 595 after being post-processed through a deblocking unit570 and a loop filtering unit 580. Also, the image data that ispost-processed through the deblocking unit 570 and the loop filteringunit 580 may be output as the reference frame 585.

In order to decode the image data in the decoder 95 of the videodecoding apparatus 90, the image decoder 500 may perform operations thatare performed after the parser 510.

In order for the image decoder 500 to be applied in the video encodingapparatus 90, elements of the image decoder 500, i.e., the parser 510,the entropy decoder 520, the inverse quantizer 530, the inversetransformer 540, the intra predictor 550, the motion compensator 560,the deblocking unit 570, and the loop filtering unit 580, performoperations based on coding units having a tree structure for eachmaximum coding unit.

Specifically, the intra prediction 550 and the motion compensator 560perform operations based on partitions and a prediction mode for each ofthe coding units having a tree structure, and the inverse transformer540 perform operations based on a size of a transformation unit for eachcoding unit.

FIG. 13 is a diagram illustrating deeper coding units according todepths, and partitions, according to an exemplary embodiment.

The video encoding apparatus 80 and the video decoding apparatus 90 usehierarchical coding units so as to consider characteristics of an image.A maximum height, a maximum width, and a maximum depth of coding unitsmay be adaptively determined according to the characteristics of theimage, or may be differently set by a user. Sizes of deeper coding unitsaccording to depths may be determined according to the predeterminedmaximum size of the coding unit.

In a hierarchical structure 600 of coding units, according to anexemplary embodiment, the maximum height and the maximum width of thecoding units are each 64, and the maximum depth is 4. Since a depthdeepens along a vertical axis of the hierarchical structure 600, aheight and a width of the deeper coding unit are each split. Also, aprediction unit and partitions, which are bases for prediction encodingof each deeper coding unit, are shown along a horizontal axis of thehierarchical structure 600.

In other words, a coding unit 610 is a maximum coding unit in thehierarchical structure 600, wherein a depth is 0 and a size, i.e., aheight by width, is 64×64. The depth deepens along the vertical axis,and a coding unit 620 having a size of 32×32 and a depth of 1, a codingunit 630 having a size of 16×16 and a depth of 2, a coding unit 640having a size of 8×8 and a depth of 3, and a coding unit 650 having asize of 4×4 and a depth of 4 exist. The coding unit 650 having the sizeof 4×4 and the depth of 4 is a minimum coding unit.

The prediction unit and the partitions of a coding unit are arrangedalong the horizontal axis according to each depth. In other words, ifthe coding unit 610 having the size of 64×64 and the depth of 0 is aprediction unit, the prediction unit may be split into partitionsincluded in the encoding unit 610, i.e., a partition 610 having a sizeof 64×64, partitions 612 having the size of 64×32, partitions 614 havingthe size of 32×64, or partitions 616 having the size of 32×32.

Similarly, a prediction unit of the coding unit 620 having the size of32×32 and the depth of 1 may be split into partitions included in thecoding unit 620, i.e., a partition 620 having a size of 32×32,partitions 622 having a size of 32×16, partitions 624 having a size of16×32, and partitions 626 having a size of 16×16.

Similarly, a prediction unit of the coding unit 630 having the size of16×16 and the depth of 2 may be split into partitions included in thecoding unit 630, i.e., a partition having a size of 16×16 included inthe coding unit 630, partitions 632 having a size of 16×8, partitions634 having a size of 8×16, and partitions 636 having a size of 8×8.

Similarly, a prediction unit of the coding unit 640 having the size of8×8 and the depth of 3 may be split into partitions included in thecoding unit 640, i.e., a partition having a size of 8×8 included in thecoding unit 640, partitions 642 having a size of 8×4, partitions 644having a size of 4×8, and partitions 646 having a size of 4×4.

The coding unit 650 having the size of 4×4 and the depth of 4 is theminimum coding unit and a coding unit of the lowermost depth. Aprediction unit of the coding unit 650 may be only assigned to apartition having a size of 4×4. Alternatively, partitions 652 having asize of 4×2, partitions 654 having a size of 2×4, or partitions 656having a size of 2×2 may be used.

In order to determine the at least one coded depth of the coding unitsconstituting the maximum coding unit 610, the coded depth and encodingmode determiner 83 of the video encoding apparatus 80 performs encodingfor coding units corresponding to each depth included in the maximumcoding unit 610.

A number of deeper coding units according to depths including data inthe same range and the same size increases as the depth deepens. Forexample, four coding units corresponding to a depth of 2 are required tocover data that is included in one coding unit corresponding to a depthof 1. Accordingly, in order to compare encoding results of the same dataaccording to depths, the coding unit corresponding to the depth of 1 andfour coding units corresponding to the depth of 2 are each encoded.

In order to perform encoding for a current depth from among the depths,a least encoding error may be selected for the current depth byperforming encoding for each prediction unit in the coding unitscorresponding to the current depth, along the horizontal axis of thehierarchical structure 600. Alternatively, the minimum encoding errormay be searched for by comparing the least encoding errors according todepths, by performing encoding for each depth as the depth deepens alongthe vertical axis of the hierarchical structure 600. A depth and apartition having the minimum encoding error in the coding unit 610 maybe selected as the coded depth and a partition type of the coding unit610.

FIG. 14 is a diagram for describing a relationship between a coding unit710 and transformation units 720, according to an exemplary embodiment.

The video encoding apparatus 80 or the video decoding apparatus 90encodes or decodes an image according to coding units having sizessmaller than or equal to a maximum coding unit for each maximum codingunit. Sizes of transformation units for transformation during encodingmay be selected based on data units that are not larger than acorresponding coding unit.

For example, in the video encoding apparatus 80 or the video decodingapparatus 90, if a size of the coding unit 710 is 64×64, transformationmay be performed by using the transformation units 720 having a size of32×32.

Also, data of the coding unit 710 having the size of 64×64 may beencoded by performing the transformation on each of the transformationunits having the size of 32×32, 16×16, 8×8, and 4×4, which are smallerthan 64×64, and then a transformation unit having the least coding errormay be selected.

FIG. 15 is a diagram for describing encoding information of coding unitscorresponding to a coded depth, according to an exemplary embodiment.

The transmitter 88 of the video encoding apparatus 80 may encode andtransmit information 800 about a partition type, information 810 about aprediction mode, and information 820 about a size of a transformationunit for each coding unit corresponding to a coded depth, as informationabout an encoding mode.

The information 800 indicates information about a shape of a partitionobtained by splitting a prediction unit of a current coding unit,wherein the partition is a data unit for prediction encoding the currentcoding unit. For example, a current coding unit CU_0 having a size of2N×2N may be split into any one of a partition 802 having a size of2N×2N, a partition 804 having a size of 2N×N, a partition 806 having asize of N×2N, and a partition 808 having a size of N×N. Here, theinformation 800 about a partition type is set to indicate one of thepartition 804 having a size of 2N×N, the partition 806 having a size ofN×2N, and the partition 808 having a size of N×N.

The information 810 indicates a prediction mode of each partition. Forexample, the information 810 may indicate a mode of prediction encodingperformed on a partition indicated by the information 800, i.e., anintra mode 812, an inter mode 814, or a skip mode 816.

The information 820 indicates a transformation unit to be based on whentransformation is performed on a current coding unit. For example, thetransformation unit may be a first intra transformation unit 822, asecond intra transformation unit 824, a first inter transformation unit826, or a second intra transformation unit 828.

The information extractor 93 of the video decoding apparatus 90 mayextract and use the information 800, 810, and 820 for decoding,according to each deeper coding unit.

FIG. 16 is a diagram of deeper coding units according to depths,according to an exemplary embodiment.

Split information may be used to indicate a change of a depth. The spiltinformation indicates whether a coding unit of a current depth is splitinto coding units of a lower depth.

A prediction unit 910 for prediction encoding a coding unit 900 having adepth of 0 and a size of 2N_(—)0×2N_(—)0 may include partitions of apartition type 912 having a size of 2N_(—)0×2N_(—)0, a partition type914 having a size of 2N_(—)0×N_(—)0, a partition type 916 having a sizeof N_(—)0×2N_(—)0, and a partition type 918 having a size ofN_(—)0×N_(—)0. FIG. 9 only illustrates the partition types 912 through918 which are obtained by symmetrically splitting the prediction unit910, but a partition type is not limited thereto, and the partitions ofthe prediction unit 910 may include asymmetrical partitions, partitionshaving a predetermined shape, and partitions having a geometrical shape.

Prediction encoding is repeatedly performed on one partition having asize of 2N_(—)0×2N_(—)0, two partitions having a size of 2N_(—)0×N_(—)0,two partitions having a size of N_(—)0×2N_(—)0, and four partitionshaving a size of N_(—)0×N_(—)0, according to each partition type. Theprediction encoding in an intra mode and an inter mode may be performedon the partitions having the sizes of 2N_(—)0×2N_(—)0, N_(—0×2)N_(—)0,2N_(—)0×N_(—)0, and N_(—)0×N_(—)0. The prediction encoding in a skipmode is performed only on the partition having the size of2N_(—)0×2N_(—)0.

Errors of encoding including the prediction encoding in the partitiontypes 912 through 918 are compared, and the least encoding error isdetermined among the partition types. If an encoding error is smallestin one of the partition types 912 through 916, the prediction unit 910may not be split into a lower depth.

If the encoding error is the smallest in the partition type 918, a depthis changed from 0 to 1 to split the partition type 918 in operation 920,and encoding is repeatedly performed on coding units 930 having a depthof 2 and a size of N_(—)0×N_(—)0 to search for a minimum encoding error.

A prediction unit 940 for prediction encoding the coding unit 930 havinga depth of 1 and a size of 2N_(—)1×2N_(—)1 (=N_(—)0×N_(—)0) may includepartitions of a partition type 942 having a size of 2N_(—)1×2N_(—)1, apartition type 944 having a size of 2N_(—)1×N_(—)1, a partition type 946having a size of N_(—)1×2N_(—)1, and a partition type 948 having a sizeof N_(—)1×N_(—)1.

If an encoding error is the smallest in the partition type 948, a depthis changed from 1 to 2 to split the partition type 948 in operation 950,and encoding is repeatedly performed on coding units 960, which have adepth of 2 and a size of N_(—)2×N_(—)2 to search for a minimum encodingerror.

When a maximum depth is d, split operation according to each depth maybe performed up to when a depth becomes d−1, and split information maybe encoded as up to when a depth is one of 0 to d−2. In other words,when encoding is performed up to when the depth is d−1 after a codingunit corresponding to a depth of d−2 is split in operation 970, aprediction unit 990 for prediction encoding a coding unit 980 having adepth of d−1 and a size of 2N_(d−1)×2N_(d−1) may include partitions of apartition type 992 having a size of 2N_(d−1)×2N_(d−1), a partition type994 having a size of 2N_(d−1)×N_(d−1), a partition type 996 having asize of N_(d−1)×2N_(d−1), and a partition type 998 having a size ofN_(d−1)×N_(d−1).

Prediction encoding may be repeatedly performed on one partition havinga size of 2N_(d−1)×2N_(d−1), two partitions having a size of2N_(d−1)×N_(d−1), two partitions having a size of N_(d−1)×2N_(d−1), fourpartitions having a size of N_(d−1)×N_(d−1) from among the partitiontypes 992 through 998 to search for a partition type having a minimumencoding error.

Even when the partition type 998 has the minimum encoding error, since amaximum depth is d, a coding unit CU_(d−1) having a depth of d−1 is nolonger split to a lower depth, and a coded depth for the coding unitsconstituting a current maximum coding unit 900 is determined to be d−1and a partition type of the current maximum coding unit 900 may bedetermined to be N_(d−1)×N_(d−1). Also, since the maximum depth is d anda minimum coding unit 980 having a lowermost depth of d−1 is no longersplit to a lower depth, split information for the minimum coding unit980 is not set.

A data unit 999 may be a ‘minimum unit’ for the current maximum codingunit. A minimum unit according to an exemplary embodiment may be arectangular data unit obtained by splitting a minimum coding unit 980 by4. By performing the encoding repeatedly, the video encoding apparatus80 may select a depth having the least encoding error by comparingencoding errors according to depths of the coding unit 900 to determinea coded depth, and set a corresponding partition type and a predictionmode as an encoding mode of the coded depth.

As such, the minimum encoding errors according to depths are compared inall of the depths of 1 through d, and a depth having the least encodingerror may be determined as a coded depth. The coded depth, the partitiontype of the prediction unit, and the prediction mode may be encoded andtransmitted as information about an encoding mode. Also, since a codingunit is split from a depth of 0 to a coded depth, only split informationof the coded depth is set to 0, and split information of depthsexcluding the coded depth is set to 1.

The information extractor 93 of the video decoding apparatus 90 mayextract and use the information about the coded depth and the predictionunit of the coding unit 900 to decode the partition 912. The videodecoding apparatus 90 may determine a depth, in which split informationis 0, as a coded depth by using split information according to depths,and use information about an encoding mode of the corresponding depthfor decoding.

FIGS. 17 through 19 are diagrams for describing a relationship betweencoding units 1010, prediction units 1060, and transformation units 1070,according to an exemplary embodiment.

The coding units 1010 are coding units having a tree structure,corresponding to coded depths determined by the video encoding apparatus80, in a maximum coding unit. The prediction units 1060 are partitionsof prediction units of each of the coding units 1010, and thetransformation units 1070 are transformation units of each of the codingunits 1010.

When a depth of a maximum coding unit is 0 in the coding units 1010,depths of coding units 1012 and 1054 are 1, depths of coding units 1014,1016, 1018, 1028, 1050, and 1052 are 2, depths of coding units 1020,1022, 1024, 1026, 1030, 1032, and 1048 are 3, and depths of coding units1040, 1042, 1044, and 1046 are 4.

In the prediction units 1060, some encoding units 1014, 1016, 1022,1032, 1048, 1050, 1052, and 1054 are obtained by splitting the codingunits in the encoding units 1010. In other words, partition types in thecoding units 1014, 1022, 1050, and 1054 have a size of 2N×N, partitiontypes in the coding units 1016, 1048, and 1052 have a size of N×2N, anda partition type of the coding unit 1032 has a size of N×N. Predictionunits and partitions of the coding units 1010 are smaller than or equalto each coding unit.

Transformation or inverse transformation is performed on image data ofthe coding unit 1052 in the transformation units 1070 in a data unitthat is smaller than the coding unit 1052. Also, the coding units 1014,1016, 1022, 1032, 1048, 1050, and 1052 in the transformation units 1070are different from those in the prediction units 1060 in terms of sizesand shapes. In other words, the video encoding and decoding apparatuses100 and 200 may perform intra prediction, motion estimation, motioncompensation, transformation, and inverse transformation individually ona data unit in the same coding unit.

Accordingly, encoding is recursively performed on each of coding unitshaving a hierarchical structure in each region of a maximum coding unitto determine an optimum coding unit, and thus coding units having arecursive tree structure may be obtained. Encoding information mayinclude split information about a coding unit, information about apartition type, information about a prediction mode, and informationabout a size of a transformation unit. Table 2 shows the encodinginformation that may be set by the video encoding and decodingapparatuses 80 and 90.

TABLE 2 Split Information 0 Split (Encoding on Coding Unit having Sizeof 2N × 2N and Current Depth of d) Information 1 Prediction PartitionType Size of Transformation Unit Repeatedly Mode Encode IntraSymmetrical Asymmetrical Split Split Coding Units Inter PartitionPartition Information 0 of Information 1 of having Skip Type TypeTransformation Transformation Lower Depth (Only Unit Unit of d + 1 2N ×2N) 2N × 2N 2N × nU 2N × 2N N × N 2N × N 2N × nD (Symmetrical N × 2N nL× 2N Type) N × N nR × 2N N/2 × N/2 (Asymmetrical Type)

The transmitter 88 of the video encoding apparatus 80 may output theencoding information about the coding units having a tree structure, andthe information extractor 93 of the video decoding apparatus 90 mayextract the encoding information about the coding units having a treestructure from a received bitstream.

Split information indicates whether a current coding unit is split intocoding units of a lower depth. If split information of a current depth dis 0, a depth, in which a current coding unit is no longer split into alower depth, is a coded depth, and thus information about a partitiontype, prediction mode, and a size of a transformation unit may bedefined for the coded depth. If the current coding unit is further splitaccording to the split information, encoding is independently performedon four split coding units of a lower depth.

A prediction mode may be one of an intra mode, an inter mode, and a skipmode. The intra mode and the inter mode may be defined in all partitiontypes, and the skip mode may be defined only in a partition type havinga size of 2N×2N.

The information about the partition type may indicate symmetricalpartition types having sizes of 2N×2N, 2N×N, N×2N, and N×N, which areobtained by symmetrically splitting a height or a width of a predictionunit, and asymmetrical partition types having sizes of 2N×nU, 2N×nD,nL×2N, and nR×2N, which are obtained by asymmetrically splitting theheight or the width of the prediction unit. The asymmetrical partitiontypes having the sizes of 2N×nU and 2N×nD may be respectively obtainedby splitting the height of the prediction unit in 1:3 and 3:1, and theasymmetrical partition types having the sizes of nL×2N and nR×2N may berespectively obtained by splitting the width of the prediction unit in1:3 and 3:1

The size of the transformation unit may be set to be two types in theintra mode and two types in the inter mode. In other words, if splitinformation of the transformation unit is 0, the size of thetransformation unit may be 2N×2N, which is the size of the currentcoding unit. If split information of the transformation unit is 1, thetransformation units may be obtained by splitting the current codingunit. Also, if a partition type of the current coding unit having thesize of 2N×2N is a symmetrical partition type, a size of atransformation unit may be N×N, and if the partition type of the currentcoding unit is an asymmetrical partition type, the size of thetransformation unit may be N/2×N/2.

The encoding information about coding units having a tree structure mayinclude at least one of a coding unit corresponding to a coded depth, aprediction unit, and a minimum unit. The coding unit corresponding tothe coded depth may include at least one of a prediction unit and aminimum unit containing the same encoding information.

Accordingly, it is determined whether adjacent data units are includedin the same coding unit corresponding to the coded depth by comparingencoding information of the adjacent data units. Also, a correspondingcoding unit corresponding to a coded depth is determined by usingencoding information of a data unit, and thus a distribution of codeddepths in a maximum coding unit may be determined.

Accordingly, if a current coding unit is predicted based on encodinginformation of adjacent data units, encoding information of data unitsin deeper coding units adjacent to the current coding unit may bedirectly referred to and used.

Alternatively, if a current coding unit is predicted based on encodinginformation of adjacent data units, data units adjacent to the currentcoding unit are searched using encoded information of the data units,and the searched adjacent coding units may be referred for predictingthe current coding unit.

FIG. 20 is a diagram for describing a relationship between a codingunit, a prediction unit or a partition, and a transformation unit,according to encoding mode information of Table 2.

A maximum coding unit 1300 includes coding units 1302, 1304, 1306, 1312,1314, 1316, and 1318 of coded depths. Here, since the coding unit 1318is a coding unit of a coded depth, split information may be set to 0.Information about a partition type of the coding unit 1318 having a sizeof 2N×2N may be set to be one of a partition type 1322 having a size of2N×2N, a partition type 1324 having a size of 2N×N, a partition type1326 having a size of N×2N, a partition type 1328 having a size of N×N,a partition type 1332 having a size of 2N×nU, a partition type 1334having a size of 2N×nD, a partition type 1336 having a size of nL×2N,and a partition type 1338 having a size of nR×2N.

Split information (TU size flag) of a transformation unit is a type oftransformation index, wherein a size of a transformation unitcorresponding to a transformation index may change according to aprediction unit type or a partition type of a coding unit.

For example, when the partition type is set to be symmetrical, i.e., thepartition type 1322, 1324, 1326, or 1328, a transformation unit 1342having a size of 2N×2N is set if TU size flag is 0, and a transformationunit 1344 having a size of N×N is set if a TU size flag is 1.

When the partition type is set to be asymmetrical, i.e., the partitiontype 1332, 1334, 1336, or 1338, a transformation unit 1352 having a sizeof 2N×2N is set if a TU size flag is 0, and a transformation unit 1354having a size of N/2×N/2 is set if a TU size flag is 1.

Referring to FIG. 18, the TU size flag is a flag having a value or 0 or1, but the TU size flag is not limited to 1 bit, and a transformationunit may be hierarchically split having a tree structure while the TUsize flag increases from 0.

In this case, the size of a transformation unit that has been actuallyused may be expressed by using a TU size flag of a transformation unit,according to an exemplary embodiment, together with a maximum size andminimum size of the transformation unit. According to an exemplaryembodiment, the video encoding apparatus 80 is capable of encodingmaximum transformation unit size information, minimum transformationunit size information, and a maximum TU size flag. The result ofencoding the maximum transformation unit size information, the minimumtransformation unit size information, and the maximum TU size flag maybe inserted into an SPS. According to an exemplary embodiment, the videodecoding apparatus 90 may decode video by using the maximumtransformation unit size information, the minimum transformation unitsize information, and the maximum TU size flag.

For example, if the size of a current coding unit is 64×64 and a maximumtransformation unit size is 32×32, then the size of a transformationunit may be 32×32 when a TU size flag is 0, may be 16×16 when the TUsize flag is 1, and may be 8×8 when the TU size flag is 2.

As another example, if the size of the current coding unit is 32×32 anda minimum transformation unit size is 32×32, then the size of thetransformation unit may be 32×32 when the TU size flag is 0. Here, theTU size flag cannot be set to a value other than 0, since the size ofthe transformation unit cannot be less than 32×32.

As another example, if the size of the current coding unit is 64×64 anda maximum TU size flag is 1, then the TU size flag may be 0 or 1. Here,the TU size flag cannot be set to a value other than 0 or 1.

Thus, if it is defined that the maximum TU size flag is‘MaxTransformSizeIndex’, a minimum transformation unit size is‘MinTransformSize’, and a transformation unit size is ‘RootTuSize’ whenthe TU size flag is 0, then a current minimum transformation unit size‘CurrMinTuSize’ that can be determined in a current coding unit, may bedefined by Equation (10):

CurrMinTuSize=max(MinTransformSize,RootTuSize/(2̂MaxTransformSizeIndex))  (10).

Compared to the current minimum transformation unit size ‘CurrMinTuSize’that can be determined in the current coding unit, a transformation unitsize ‘RootTuSize’ when the TU size flag is 0 may denote a maximumtransformation unit size that can be selected in the system. In Equation(10), ‘RootTuSize/(2̂MaxTransformSizeIndex)’ denotes a transformationunit size when the transformation unit size ‘RootTuSize’, when the TUsize flag is 0, is split a number of times corresponding to the maximumTU size flag, and ‘MinTransformSize’ denotes a minimum transformationsize. Thus, a smaller value from among‘RootTuSize/(2̂MaxTransformSizeIndex)’ and ‘MinTransformSize’ may be thecurrent minimum transformation unit size ‘CurrMinTuSize’ that can bedetermined in the current coding unit.

According to an exemplary embodiment, the maximum transformation unitsize RootTuSize may vary according to the type of a prediction mode.

For example, if a current prediction mode is an inter mode, then‘RootTuSize’ may be determined by using Equation (11) below. In Equation(11), ‘MaxTransformSize’ denotes a maximum transformation unit size, and‘PUSize’ denotes a current prediction unit size.

RootTuSize=min(MaxTransformSize,PUSize)  (11).

That is, if the current prediction mode is the inter mode, thetransformation unit size ‘RootTuSize’ when the TU size flag is 0, may bea smaller value from among the maximum transformation unit size and thecurrent prediction unit size.

If a prediction mode of a current partition unit is an intra mode,‘RootTuSize’ may be determined by using Equation (12) below. In Equation(12), ‘PartitionSize’ denotes the size of the current partition unit.

RootTuSize=min(MaxTransformSize,PartitionSize)  (12).

That is, if the current prediction mode is the intra mode, thetransformation unit size ‘RootTuSize’ when the TU size flag is 0 may bea smaller value from among the maximum transformation unit size and thesize of the current partition unit.

However, the current maximum transformation unit size ‘RootTuSize’ thatvaries according to the type of a prediction mode in a partition unit isjust an example and the present invention is not limited thereto.

FIG. 21 is a flowchart illustrating a method of encoding a video forcompensating for a pixel value after performing loop filtering based oncoding units having a tree structure, according to an exemplaryembodiment.

In operation 2110, a current picture is split into at least one maximumcoding unit, and a coded depth to output a final encoding resultaccording to at least one split region, which is obtained by splitting aregion of each maximum coding unit according to depths, is determined byencoding the at least one split region. Also, an encoding mode includinginformation about a coded depth or split information, information abouta partition type of a coded depth, a prediction mode, and a size of atransformation unit, is determined according to a deeper coding unitaccording to depths.

A maximum depth indicating a total number of possible times the maximumcoding unit is split may be predetermined. The maximum coding unit maybe hierarchically split, and encoding may be repeatedly performed foreach deeper coding unit whenever the depth deepens. Encoding errors ofall deeper coding units are measured and compared so as to determine acoded depth that generates the least encoding error of the coding unit.

In operation 2120, encoded image data is decoded based on the codeddepth and the encoding mode, and a restored image is generated byperforming loop filtering on the decoded image data. The restored imagemay be generated by performing adaptive loop filtering, whichcontinuously performs at least one one-dimensional filtering on thedecoded image data or deblocked image data.

In operation 2130, a compensation value about an error between eachrestored pixel in a predetermined group of the restored image and anoriginal pixel, and a pixel group including restored pixels to becompensated for are determined. The pixel group including the restoredpixels having pixel values to be compensated for may be determinedaccording to extreme and/or edge value levels of pixel values, bands ofpixel values, or lines. The compensation value according to pixel groupsmay be determined based on an average value of the errors.

In operation 2140, image data constituting the final encoding resultaccording to the at least one split region, information about the codeddepth and the encoding mode, information about a loop filteringcoefficient, and information related to the compensation value areoutput. The information about the encoding mode may include informationabout a coded depth or split information, information about a partitiontype of the coded depth, a prediction mode, and a size of atransformation unit.

The information related to the compensation value according to the pixelgroups may be encoded along with the information about the encodingmode, the video data, and the information about the loop filteringcoefficient, which are encoded according to the method based on thecoding units having the tree structure, and may be transmitted to adecoder.

FIG. 22 is a flowchart illustrating a method of decoding a video forcompensating for a pixel value after performing loop filtering based oncoding units having a tree structure, according to an exemplaryembodiment.

In operation 2210, a bitstream about a video encoded according to themethod of FIG. 21 based on the coding units having the tree structure isreceived and parsed, and image data of a current picture assigned to amaximum coding unit, information about a coded depth and an encodingmode according to the maximum coding units, information about a loopfiltering coefficient, and information related to a compensation valueare extracted from the parsed bitstream.

A coded depth according to the maximum coding units is selected as adepth having the least encoding error according to maximum coding unitswhile encoding the current picture. Encoding is performed according tothe maximum coding units by encoding the image data based on at leastone data unit obtained by hierarchically splitting the maximum codingunit according to depths. Accordingly, each piece of image data isdecoded after determining the coded depth according to coding units,thereby improving encoding and decoding efficiency of an image.

In operation 2220, the image data is decoded in each maximum coding unitbased on the information about the coded depth and the encoding mode,and a restored image is generated by performing loop filtering on thedecoded image data. The restored image may be generated by performingadaptive loop filtering, wherein at least one one-dimensional filteringis continuously performed, on the decoded image data or deblocked imagedata.

In operation 2230, a pixel group including restored pixels to becompensated for is determined from among restored pixels of the restoredimage, by using the compensation value. The pixel group including therestored pixels having pixel values to be compensated for may bedetermined by using the compensation value, according to extreme and/oredge value levels of the pixel values of the restored pixels, bands ofpixel values, or lines, by using a method of determining a pixel groupbased on the extracted information related to the compensation value.

In operation 2240, a restored image having a compensated error may beoutput by compensating an error between the restored pixels of thedetermined pixel group and corresponding original pixel by using thecompensation value.

According to the method of encoding a video and the method of decoding avideo, the quality of the restored image may be improved by compensatingfor a systematic error of the restored image, and a transmission bitrate of additional information for improving the quality of the restoredimage may be reduced since only the information about the compensationvalue according to the pixel groups is encoded and transmitted, andinformation about a location of a pixel to be compensated for is nottransmitted.

Exemplary embodiments of the present invention can be written ascomputer programs and can be implemented in general-use digitalcomputers that execute the programs using a computer readable recordingmedium. Examples of the computer readable recording medium includemagnetic storage media (e.g., ROM, floppy disks, hard disks, etc.) andoptical recording media (e.g., CD-ROMs, or DVDs).

While exemplary embodiments have been particularly shown and describedabove, it will be understood by those of ordinary skill in the art thatvarious changes in form and details may be made therein withoutdeparting from the spirit and scope of the inventive concept as definedby the appended claims. The exemplary embodiments should be consideredin a descriptive sense only and not for purposes of limitation.Therefore, the scope of the inventive concept is defined not by thedetailed description of exemplary embodiments but by the appendedclaims, and all differences within the scope will be construed as beingincluded in the present inventive concept.

1. A method of encoding a video for compensating for a pixel value, themethod comprising: encoding image data; decoding the encoded image dataand generating a restored image by performing loop filtering on thedecoded image data; determining a compensation value corresponding toerrors between a predetermined group of restored pixels in the restoredimage and corresponding original pixels, and a pixel group comprising arestored pixel to be compensated for by using the compensation value;and encoding the compensation value and transmitting the encodedcompensation value and a bitstream of the encoded image data.
 2. Themethod of claim 1, wherein the determining the compensation value andthe pixel group comprises: determining an extreme value level indicatinga degree of closeness to a maximum value or a minimum value according torestored pixels by comparing pixel values of neighboring restored pixelsin the restored image; and determining the pixel group comprising therestored pixel to be compensated for among the neighboring restoredpixels based on the determined extreme value level, wherein the extremevalue level includes edge value level among the pixel values ofneighboring restored pixels in the restored image.
 3. The method ofclaim 2, wherein: the determining the pixel group based on the extremevalue level comprises classifying the neighboring restored pixels intopixel groups comprising restored pixels having a same extreme valuelevel based on the determined extreme value level, and determining apixel group of at least one extreme value level as the pixel groupcomprising the restored pixel to be compensated for, and the determiningthe compensation value and the pixel group further comprises determiningthe compensation value for the determined pixel group of the at leastone extreme value level.
 4. The method of claim 1, wherein thedetermining the compensation value and the pixel group comprises:classifying restored pixels of the restored image into pixel groupscomprising restored pixels in a same band, according to bands, obtainedby dividing a total section of pixel values; and determining thecompensation value according to the pixel groups according to bands,wherein the bands are uniform or non-uniformly split based on actualminimum and maximum values of original pixels and restored pixels. 5.The method of claim 4, wherein the classifying the restored pixelsaccording to bands comprises classifying the restored pixels into thepixel groups according to bands, based on bands obtained by dividing thetotal section of the pixel values into a number of 2 to the power of apositive integer.
 6. The method of claim 5, wherein an index of thenumber of 2 to the power of the positive integer is determined based ona number of most significant bits in a bit depth of the restored pixels.7. The method of claim 1, wherein the determining the compensation valueand the pixel group comprises: classifying restored pixels of therestored image into pixel groups comprising restored pixels disposed ona same line, according to lines; and determining the compensation valueaccording to the pixel groups according to lines.
 8. The method of claim7, wherein the classifying the restored pixels into the pixel groupsaccording to lines comprises detecting restored pixels forming lines inat least one of a horizontal direction, a vertical direction, a diagonaldirection, a curved direction and a boundary direction of apredetermined object, from among the restored pixels of the restoredimage.
 9. The method of claim 1, wherein the determining thecompensation value and the pixel group comprises determining thecompensation value by using an average value of errors between restoredpixels of the pixel group and corresponding original pixels.
 10. Themethod of claim 1, wherein the generating the restored image comprisesgenerating the restored image using adaptive loop filtering using aplurality of continuous one-dimensional filters.
 11. The method of claim1, wherein the determining the compensation value and the pixel groupcomprises determining the compensation value and the restored pixel tobe compensated for according to at least one data unit from among animage sequence, a slice, a frame, and a coding unit of the video. 12.The method of claim 1, wherein the transmitting the bitstream comprisesinserting and transmitting the encoded compensation value andinformation regarding the minimum and maximum values of original pixelsand adjusting bands based on the actual minimum and maximum values ofthe original pixels, into a slice header.
 13. The method of claim 1,wherein the encoding the image data comprises: dividing a picture into amaximum coding unit; performing encoding on at least one deeper codingunit according to depth, according to regions split by hierarchicallydividing the maximum coding unit as the depth deepens to determine anencoding mode of a coding unit of a coded depth, the encoding modecomprising information of at least one coded depth that generates aleast encoding error; and outputting the encoded image data according tothe determined coded depth and encoding mode.
 14. A method of decoding avideo for compensating for a pixel value, the method comprising:extracting encoded image data and a compensation value from a bitstreamby parsing the bitstream of an encoded image; decoding the extractedimage data and generating a restored image by performing loop filteringon the decoded image data; determining a pixel group comprising arestored pixel to be compensated for from among restored pixels of therestored image, by using the extracted compensation value; andcompensating for an error between the restored pixel of the determinedpixel group and a corresponding original pixel by using the extractedcompensation value.
 15. The method of claim 14, wherein the determiningthe pixel group comprises: determining an extreme value level, whichindicates a degree of closeness to a maximum value or a minimum valueaccording to the restored pixels, by comparing pixel values ofneighboring restored pixels in the restored image; and determining thepixel group comprising the restored pixel to be compensated for amongthe neighboring restored pixels based on the determined extreme valuelevel, wherein the extreme value level includes edge value level amongthe pixel values of neighboring restored pixels in the restored image.16. The method of claim 15, wherein the determining the extreme valuelevel comprises classifying the neighboring restored pixels into pixelgroups comprising restored pixels having a same extreme value levelbased on the determined extreme value level, and determining a pixelgroup of at least one extreme value level as the pixel group comprisingthe restored pixel to be compensated for.
 17. The method of claim 16,wherein the compensating for the error comprises compensating for errorsof pixel values of the restored pixels of the pixel groups according toextreme value levels by using the compensation values according toextreme value levels for compensating for the pixel values of the pixelgroups according to the extreme value levels.
 18. The method of claim14, wherein the determining the pixel group comprises classifying therestored pixels of the restored image into pixel groups comprisingrestored pixels in a same band, according to bands, obtained by dividinga total section of pixel values, and determining a pixel group, fromamong the pixel groups according to bands, as the pixel groupscomprising the restored pixel to be compensated for.
 19. The method ofclaim 18, wherein the compensating for the error comprises compensatingfor pixel values of the restored pixels of the pixel groups according tobands, by using compensation values according to bands for the pixelgroups according to bands, wherein the bands are uniform ornon-uniformly split based on actual minimum and maximum values oforiginal pixels and restored pixels.
 20. The method of claim 18, whereinthe classifying the restored pixels into the pixel groups according tobands comprises classifying the restored pixels into the pixel groupsaccording to bands, based on bands obtained by dividing the totalsection of the pixel values into a number of 2 to the power of apositive integer.
 21. The method of claim 20, wherein an index of thenumber of 2 to the power of the positive integer is determined based ona number of most significant bits in a bit depth of the restored pixels.22. The method of claim 14, wherein the determining the pixel groupcomprises classifying the restored pixels of the restored image intopixel groups comprising restored pixels disposed on a same line,according to lines, and determining a pixel group, from among the pixelgroups according to lines, as the pixel group comprising the restoredpixel to be compensated for.
 23. The method of claim 22, wherein thecompensating for the error comprises compensating for pixel values ofthe restored pixels in the pixel groups according to lines, by usingcompensation values of the pixel groups according to lines forcompensating for the pixel values of the pixel groups according tolines.
 24. The method of claim 22, wherein the classifying the restoredpixels into the pixel groups according to lines comprises detectingrestored pixels forming lines in at least one of a horizontal direction,a vertical direction, a diagonal direction, a curved direction and aboundary direction of a predetermined object, from among the restoredpixels of the restored image.
 25. The method of claim 14, wherein thecompensation value is determined by using an average value of errorsbetween the restored pixels of the pixel group and correspondingoriginal pixels during encoding of the encoded image data.
 26. Themethod of claim 14, wherein the generating the restored image comprisesgenerating the restored image using adaptive loop filtering using aplurality of continuous one-dimensional filters.
 27. The method of claim14, wherein the extracting comprises extracting the compensation valueaccording to at least one data unit from among an image sequence, aslice, a frame, and a coding unit of the input video.
 28. The method ofclaim 14, wherein the extracting comprises extracting the compensationvalue and information regarding minimum and maximum values of originalpixels, from a slice header of the bitstream.
 29. The method of claim14, wherein: the encoded image data is encoded by dividing a pictureinto a maximum coding unit, and performing encoding on at least onedeeper coding unit according to depth, according to regions split byhierarchically dividing the maximum coding unit as the depth deepens todetermine information of an encoding mode of a coding unit of a codeddepth, which comprises information about at least one coded depth thatgenerates a least encoding error, from the bitstream; and the generatingthe restored image comprises decoding the image data encoded based onthe coded depth and the encoding mode, based on the information aboutthe encoding mode, and performing the loop filtering.
 30. An apparatusfor encoding a video for compensating for a pixel value, the apparatuscomprising: an encoder which encodes image data; a restored imagegenerator which decodes the encoded image data and generates a restoredimage by performing loop filtering on the decoded image data; acompensation value and pixel group determiner which determines acompensation value corresponding to errors between a predetermined groupof restored pixels in the restored image and corresponding originalpixels, and a pixel group comprising a restored pixel to be compensatedfor by using the compensation value; and a transmitter which encodes thecompensation value and transmits the encoded compensation value and abitstream of the encoded image data.
 31. An apparatus for decoding avideo for compensating for a pixel value, the apparatus comprising: anextractor which extracts encoded image data and a compensation valuefrom a bitstream by parsing the bitstream of an encoded image; arestored image generator which decodes the extracted image data andgenerates a restored image by performing loop filtering on the decodedimage data; a pixel group determiner which determines a pixel groupcomprising a restored pixel to be compensated for from among restoredpixels of the restored image, by using the extracted compensation value;and a restored pixel compensator which compensates for an error betweenthe restored pixel of the determined pixel group and a correspondingoriginal pixel by using the extracted compensation value.
 32. A computerreadable recording medium having recorded thereon a program forexecuting the method of claim
 1. 33. A computer readable recordingmedium having recorded thereon a program for executing the method ofclaim 14.